The
Future
Database

The most exciting advancements in DevOps have been those that freed developers from complex operations that overwhelmed their day-to-days. This has revolutionized development workflows. Engineering teams are now built around product and business goals, not maintenance and servicing. But databases have not caught up. Product teams still need to wait on the experts to perform overly complicated database operations tasks.

The Future Database has to fit into everyone’s DevOps workflows. It must be usable by any developer and leveraged by every team within a technical org. It won’t be gated or ‘hands-off’ but built with cross-functionality in mind. How? With a user experience fluid enough to be understood by casual data consumers and mastered by data admins. The database needs to be safe enough to be operable by everyone, and do the right thing when something goes wrong — swap 2 AM pages for more rollbacks and errors caught as part of the standard workflow.

With the Future Database, database admins will no longer be required to certify their knowledge in backups and capacity planning, or worry over disaster recovery. Developer-friendly features like database branching, database developer environments, autoscaling, and usage-based billing will just be commonplace.

Some database companies are focused on building future databases around ease-of-use and translatability from Javascript. Those are enticing features. But some of those solutions sacrifice reliability and safety to do that. As billions of people around the world do more, businesses are grappling with how to process the unprecedented increase in transactions.

Apps that worked fine a few years ago are hitting new scaling limits. The databases that will fuel the next era of digital transactions must be battle-tested and proven at ridiculously high demands. A database is nothing without trust. With engineers being asked to guarantee uptime as their apps quickly scale, there might be a temptation to make concessions around reliability. But you shouldn’t have to choose. Future databases should respect ACID, be biased towards consistency over translatability and put trustworthiness at the forefront.

SQL languages will continue to win over converts as their interfaces become slicker. They’ll also become more powerful in the future as innovations like cross-shard transactions, query replication and safety rankings become standard trade.

Given the demands on the Future Database, there’s only one way to ensure it’s easy to operate while being fundamentally reliable. Make it serverless by default. Serverless is currently going through a redefinition from “functions in the cloud” and is becoming the standard for fast-paced development teams. Teams that don’t want to worry about vCPUs or node pools, but delivering outcomes.

That’s not surprising. Teams want to pay only for what they use. They want their cloud infrastructure to be available when a user needs it, scaling linearly as needed. In the future, teams won’t have to worry about capacity planning or have a machine always running in the background at a fixed instance size. Developers of the future can expect transparency in their tools. Or at least to be told what is being abstracted. Serverless gives developers this highly tuned experience – while also giving them well-designed escape hatches.

It’s a given that Future Databases will be accessed by every imaginable client – from toasters to fleets of autonomous drones. But this client-agnostic approach requires Future Databases to plan for many more transactions and connections. More than many existing databases could ever serve simultaneously.

In the future, it will be standard for databases to handle unlimited connections with smart connection pooling, query replication, and intelligent query routing. Databases will also need to serve a broad array of connection types and clients. Already, we’re seeing the exciting prospect of many more databases offering web APIs and new connection types. As these get tested at scale, we will see them adopted and used to power the next generation of applications.

Most businesses and applications today do not need a global database. It often just introduces more complexity and expense, but there are many reasons the future database will need to be globally distributed.

Being compliance-ready with regionally stored data allows you to scale rapidly to new regions. It gives your users a great experience by serving them locally available data quickly. Mobile games and other high-transaction, low-latency applications already need this. Expect more companies to require it in the future.

There’s just one caveat: global data distribution cannot break Rule 1; being easy to operate. The tools enabling this will need to be elegant and have clear interfaces. After all, their goal is to make operations easier, not harder.

The Future Database will be intelligent. It will learn and optimize processes for its developers. Imagine a database that routes queries automatically based on connection locations. Or a database that shards based on geography and will even suggest which tables to shard and where.

A future engineer will be able to optimize queries, improve database performance and even make adjustments intelligently based on recommendations from a database that learns behavior. A database designed to always do the right thing for the developer is coming.

We saved the best for last. The Future Database will first and foremost need to be prepared for future scale. Only a handful of companies on the planet have hit a scale that tests the limits of current technologies. YouTube serves an unimaginable number of queries, but, in the future, many more applications will need to scale past current records. They’ll do that by scaling both vertically and horizontally within seconds and without a human turning a knob. As it stands, many current databases require a maintenance window, replication, and a failover before they can be scaled up or out (horizontally). That downtime is unacceptable. Scale should not come at the cost of performance. And it doesn’t have to. The Future Database will scale intelligently and without a blip on the status page.