Now available from O'Reilly: the complete resource for building unkillable apps at scale with Distributed SQLGet your free copy now!
We depend on financial companies and services to help us navigate just about everything — to the point where they’re basically a form of critical infrastructure, at least existentially speaking. (Try to imagine getting through the rest of your day right now if your credit card froze or your financial services were suddenly unavailable. Do you have enough cash on you to buy lunch, or a ticket/tank of gas to drive home?). In such a complex and far-reaching sector, there are many ways things can go wrong. And, unfortunately, as your platform/application/services grow, the number of possible stress points scales right along with them.
These scale-related fail points fall into three main categories: complexity, reliability, and geo location challenges. How can fintech companies scale fast, scale globally, and above all scale reliably? Here are true tales of how three global financial services companies faced, and solved, these three different challenges.
Quickly scaling a multinational payments company with tens of millions of credit card users presents technical challenges. Here’s how one fintech company succeeded.
Nubank is a digital bank and financial services company born from founder David Vélez’s frustration over inflated lending rates and poor customer service from traditional banks in Brazil.
As a cloud-first digital bank, Nubank was able to start fast and eliminate costs associated with physical banking locations to launch a credit card with no mandatory fees and offering below-market interest rates. Within a few years Nubank had tens of millions of customers and had become the most valuable private digital bank on earth, expanding into Mexico, Argentina, and Colombia.
Nubank’s initial infrastructure was built with Java Heap using an in-memory data store and Kafka for communicating data changes. As the bank expanded numbers of both customers and countries served, this became an increasingly complex transaction environment that required constant operational babysitting. The Nubank team sought a new infrastructure that would scale elastically and automatically while still being highly available — and with minimal operational overhead.
In search of automation, Nubank migrated their credit card authorization system to CockroachDB (read the full case study). It was the team’s first choice because CockroachDB scales automatically: Data is broken into 64-megabyte ranges that distribute and balance across nodes instantly and responsively. Adding a new node simply requires spinning one up and pointing it at the cluster. They also appreciated other ease-of-operation features like automated backups, automated garbage collection, and rolling, no-downtime upgrades.
Automating scale and so many routine tasks freed up Nubank’s developer and infrastructure teams to use their time building new features and improving their apps and services — instead of bogging down in complex manual operations and maintenance.
“Regulators are saying that we are becoming critical infrastructure because of the amount of volume that we are doing.” So says Form3 Head of Platform Kevin Holditch.
Form3 is a UK-based payment technology platform that provides financial institutions with managed end-to-end payment processing. The company was established in 2016 by four banking and technology leaders with a single purpose: to transform outdated, complex and costly payments infrastructure into a modern, cloud native and, above all, real time Payments-as-a-Service solution. Form3 works with regulated financial institutions like Mastercard, Lloyd’s Banking Group,, and Square, handling 500m+ transactions (approximately $215bn) each year…And now the company is expanding from Europe into the US.
Form3’s fintech customers trust the platform to be rock-solid. However, that critical infrastructure designation did lead some to begin wondering what would happen if AWS — Form3’s cloud provider — went down.
“Ensuring reliability means we can’t be dependent on any one cloud vendor,” Holditch says. The solution? “Rather than having two data centers and one cloud, we’re going to have a Kubernetes cluster in each cloud vendor, so Azure, AWS, GCP – and a single distributed database across all three.” CockroachDB, with its support for multi-cloud and hybrid cloud deployments, was a perfect fit.
The ultimate goal, Holditch says, is to be able to tell banks that they never have to worry about any outage, anywhere around the world, slowing or stopping their increasingly global services. Because their platform will keep running even if an entire region, or even entire provider, goes down.
Imagine handling 7.2 billion API calls each month, in support of US$7.8bn in monthly transactions that take place around the globe — and ensuring data domiciling for each one of those transactions. Pismo does it. Here’s how.
There’s no question that regulatory compliance is one of the main issues fintech companies face when attempting to scale their business. Anyone from the global financial services sector can tell you that dealing with data domiciling regulations can be painfully complicated, tedious, and time consuming.
Pismo, based in Brazil and the UK, offers a fully multi-cloud SaaS platform for digital banking and payment processing. Large banks and fintech companies rely on Pismo’s platform to manage their core banking operations like card issuing, digital accounts, mobile wallets, and other next-gen payment platforms. It all adds up to 7.2 billion API calls each month in support of US$7.8bn in monthly transactions.
With customers around the world and offices in Brazil, the US, the UK, and Singapore, Pismo knows data domiciling difficulties all too well. When it came to building their global asset registration system, they simply expected reliability, availability, and guaranteed consistency as table stakes. The next-level solution they struggled to find was a horizontally scalable platform that would allow them to continue expanding all across the globe while adhering to regional, and even local, data storage regulations.
After testing CockroachDB, the team found that it could actually increase their platform’s resilience/survivability, consistency, and performance while granting the ability to control data locality at the table, or even row, level. Pismo gained powerful control over their global data’s granular location — while also maintaining a single-table developer experience.
Pismo deployed their asset registration, custody, and management services to CockroachDB Dedicated on Kubernetes to serve as their system of record. With CockroachDB, the platform is able to support intense loads of create, read, update, delete (CRUD) operations, in particular when performing asset registrations and daily accruals. This data consists of fixed income debt instruments, and a vast amount of financial data such as rates, transactions, and redemptions.
All of this data is, of course, subject to rules that governments worldwide have enacted around data domiciling to keep data close to the user and restrict access/ensure privacy. As Pismo expands globally, they need to be ready to deploy anywhere while respecting the legal obligations of data storage. Given these requirements, they are deploying CockroachDB Dedicated on Kubernetes to achieve horizontal scale and distribution.
Fintech has become completely embedded in our daily lives. Consider a typical Friday evening for many folks: First, you check your bank’s mobile app to confirm that your paycheck has landed in your account, and then consult Mint to decide your weekend entertainment budget. You head to dinner with friends, splitting the tab with Venmo, and then to a bar where you cover drinks with a quick tap on your phone’s Apple Pay app. When the evening ends, you call a Lyft and pay for the ride with a stored credit card.
Every single one of these actions are something that, as consumers, we basically take for granted…
…Which means that fintech companies can’t take anything for granted when it comes to scale, reliability and global data.
For operators, architects and developers alike, data domiciling can be one tough technical nemesis.
Privacy regulations …Read more