A 16 year payment system engineering veteran (Square, Varo, BlackRock) explains how to build payments systems that scale.Watch now
Existing traditional banking processes, interfaces, and underwriting methods don’t adequately help startup companies control their finances. Traditional banking infrastructure is too rigid. It doesn’t scale easily enough to meet the needs of startup companies that are likely to see adoption at unexpected intervals.
Tribal Credit exists to bridge the gap between what traditional financial services offer and what startups and SMBs actually need. Currently Tribal Credit has more than 2K customers worldwide. Their business started in the U.S. in 2016, but over the past few years they’ve expanded to new regions. As business grew beyond borders, their infrastructure started to reveal limitations with scale.
“Since the start of the pandemic, we’ve seen an increasing demand to shorten the distance of B2B accounts payable lifecycle, and the need for agile online payments to support this global, digital-first economy has accelerated. For us to deliver the best customer experience, we need a globally scalable and distributed platform that can support us now and grow with us in the future.” - Cesar Berho Gallardo, CTO
The team at Tribal Credit originally built their platform using SQL Server and MySQL as their primary databases. Relational database management systems (RDBMS) worked well for their initial use cases and allowed them to find product-market fit.
Since Tribal Credit’s initial version was released into the wild 6 years ago, they’ve found tremendous demand for their platform. The team grew to 120 engineers to support business momentum. High demand meant that they needed to find a way to scale without creating too much manual labor or taking on too much technical debt. Manually sharding their database and implementing synchronous replication to avoid data loss (or data inconsistencies) in the event of a failure were not acceptable paths forward.
The company also needed a globally distributed platform that was compliant with both industry requirements and region specific regulations. This required a solution that could give them the ability to replicate data across regions and control where that data resides.
Essentially Tribal Credit’s requirements evolved with their success, from a country-specific application to a globally distributed platform. This meant that they needed a database that could deliver:
They evaluated Cassandra, MongoDB, and other distributed databases for various use cases. However, they realized that they would likely have to use a combination of these databases to support different workloads, which ultimately could make their architecture even more complicated.
One of Tribal Credit’s lead software architects, Julio Monroy Quirarte, suggested that they try CockroachDB. He had heard about CockroachDB at a conference and thought that the demo was very appealing and its attributes felt like a database of the future.
“A distributed database that’s capable of performing in multi-active mode, delivers automated scale, and has the ability to replicate data across regions were important features to us. We are also fans of PostgreSQL, so finding a way to have a PostgreSQL-like database and distributed features in the form of PaaS was like a dream.” - Julio Monroy Quirarte, Software Architect
The Tribal Credit team started using CockroachDB as part of a project to split their monolithic application into microservices. They were able to get their first set of services into production within a month. Because CockroachDB is Postgres-wire compatible, they didn’t have to learn a new language and could continue using SQL.
“We were amazed by CockroachDB’s compatibility with PostgreSQL. We were expecting just some level of compatibility and a lot of constraints, however, we are able to use most of the features that we were expecting from a full-blown PostgreSQL instance. Being able to reuse our SQL knowledge is a great advantage.” - Julio Monroy Quirarte, Software Architect
Their application is written in Golang and they use Kubernetes to manage their microservices. They also use Kafka for event streaming, Redis as a data cache, and Snowflake as a data warehouse.
Currently they are running in a single AWS region (US-West-2) but they plan to expand to 3 regions in the near future to reach new markets in Latin America. One of the bonuses of working with a distributed database is that it takes care of the heavy lifting for you and you don’t have to worry about adding complexity in the application layer when you want to scale to a new region.
Using CockroachDB makes it easy for Tribal Credit to scale from one region to many with simplified multi-region deployments — basically, four settings (two are optional) that are set up with simple SQL statements. For example, when they add a second region in Sao Paulo, CockroachDB will function as a single logical database across both regions. This means that data can be accessed via any node in any region. Data is written to the region closest to the user and it is replicated across multiple nodes in each region.
If a node in a region goes down, all that region’s reads and writes will be handled by other nodes in that region. If an entire region goes down, CockroachDB will automatically spread out data across the remaining available regions.
Tribal Credit primarily handles transactional data which comes in the form of customer purchases, payments, and balances. Their traffic is steadily increasing on a daily basis, and they expect to cross the 1,000 transactions per second (TPS) mark by the end of the year.
Since Tribal Credit is handling money, their survivability goal is high. Extending their application across multiple regions serves their survival goal by ensuring high availability. Additionally, it helps maintain the integrity of their data and CockroachDB guarantees consistent transactions even in the event of a failure.
“The ability to deploy CockroachDB in multiple regions and just let it do its work has greatly simplified our architecture. We don’t need to worry about performing sharding or data replication at the application level; we can confidently deploy our platform in new regions and know our data will be in sync.” - Julio Monroy Quirarte, Software Architect
Tribal Credit’s services are currently offered in the US, Mexico, and the Middle East and North Africa (MENA). They plan to expand through Q3 and Q4 to Chile, Colombia, Peru, and Brazil in the near future. With CockroachDB, they can continue to add regions and seamlessly scale their application into new territories.
Over the next five years they envision that Tribal will be thriving in emerging markets, breaking the status quo with innovative financial solutions, and extending crypto-based solutions that will converge the best of both worlds supporting the modernization of economies.
Everybody, from SaaS applications to retailers, has to deal with payments. But architecting a system that can deal with …Read more
What do you do when you need the speed of a NoSQL database and the consistency of a relational database at the same …Read more
Nubank, a leading Brazilian financial technology company valued at more than $45 billion dollars, needed a scalable SQL …Read more