Guy Harrison is CEO of alwaysNFT.cloud, a self-funded startup that uses CockroachDB. He is the author of many books on database technology including “O’Reilly’s CockroachDB - the definitive guide”, “MongoDB Performance Tuning”, “Next Generation Databases” and “MySQL Stored Procedure Programming”. He also writes a monthly column at Database Trends and Applications. Contact Guy at firstname.lastname@example.org.
Many people believe that “Web3" is a revolution in web technology as significant as that of Web 2.0. While there’s a lot of dispute over what the term means, most agree that Web3 involves decentralized solutions, blockchains and the use of cryptocurrencies and tokens to orchestrate transactions between individuals and applications.
In today’s world, we rely on centralized organizations – Google, Twitter, Uber, Facebook, the banks, etc. to enable our digital lives – and, as a result of their incredible power, many of these centralized organizations seem to be showing a tendency toward monopolistic and antisocial behavior. In response, Web3 advocates imagine a more peer-to-peer system in which no central authority controls our interactions, powered by smart contracts and tokens living on public blockchains.
Whatever you think about Web3, there’s no disputing that there’s an enormous amount of economic value tied up in these technologies. Even in the current crypto “winter” the market capitalization of cryptocurrencies sits at more than a trillion dollars while NFTs (Non-Fungible Token) sales still exceed $100 million every day!
At first glance, transactional SQL databases might seem part of the old world and not directly relevant to Web3 development. However, there’s nothing in the Web3 stack that provides scalable, structured, and transactionally consistent data storage at scale. Blockchains simply do not offer the functionality, productivity, and economical data storage offered by distributed SQL engines such as CockroachDB. Built using distributed architecture from the ground up, CockroachDB pairs excellently with blockchain technology.
Let’s look at how CockroachDB can be used to build a Web3 application.
Today’s NFTs are largely speculative collector’s items – the equivalent of rare stamps or trading cards. While the intrinsic value of these NFTs may be nebulous, they attract very high valuations based on their exclusivity and rarity. Some of these NFTs may prove to be poor investments, but as Tomas Pueyo notes NFTs, like other luxury items, also signal to the world that the owner is wealthy and prestigious.
In the future, NFTs may serve more serious purposes – augmenting or replacing deeds and titles, for instance. Because an NFT can be transferred quickly and cheaply between two parties without the need for an intermediary adding fees and delays, many imagine them being used as ownership certificates for both digital and real-world assets.
For this Web3topia to emerge, there are many legal and technical challenges. One of the most significant technical challenges is the nature of off-chain storage. NFTs are essentially ownership certificates stored on the blockchain. However, the blockchain itself does not store the digital assets (the actual art or music files) themselves. These are stored on some off-chain storage platform. While the blockchain certificate is immutable and permanent, what stops the off-chain files from changing or disappearing?
The InterPlanetary File System (IPFS) provides a partial solution; because links in IPFS are based on hashes, the link will always resolve to the same data – if you change the data, you change the hash and hence the link. But there’s nothing to stop files in IPFS from “aging out” if they are not accessed frequently. Consequently, NFT creators should make sure that the NFT files stay “pinned” in IPFS - either on their own servers or on those of a third party. But if you are an NFT owner, how do you ensure that the NFT stays in the IPFS system? This is where alwaysNFT and CockroachDB come into play.
A few months ago, we started working on a system that would allow NFT owners to be confident that the assets associated with their NFTs would not fade away. alwaysNFT allows NFT owners to register their collections with us, and then we keep them pinned in our geo-distributed IPFS servers.
IPFS is a great solution for storing immutable blobs of data, but it’s not a database. So, we also needed a database that could keep track of all the assets we were storing in IPFS, allow fast access to the appropriate files and satisfy all the other needs of a modern, distributed web application.
Because of the huge size of the NFT market and the IPFS storage system, we needed a database that would scale and which could, if necessary, provide low latency responses to our multiple web endpoints around the globe. We also needed transactional capabilities to ensure that all our web endpoints and IPFS storage servers were in sync. We also foresaw the need to leverage the data we were collecting to refine our offering, so although we didn’t want a data warehouse as such, we did want something that could process analytical queries in real-time. And, of course, we needed high availability since the NFT ecosystem is a 24x7 concern.
If we’d been building this five years ago, we probably would have had to make some sort of awful compromise between a monolithic SQL database or a distributed NoSQL system. However, today we are fortunate enough to have distributed transactional SQL solutions such as CockroachDB. CockroachDB ticked all our boxes and became the foundation of our application.
For a web-based startup, a fully managed cloud service is a natural choice. CockroachDB can be self-hosted, but it just doesn’t make sense for a small team to dedicate resources to configuring, deploying and administering their own cluster. It’s both less expensive and less risky to use a CockroachDB managed services option.
CockroachDB managed services are available in both dedicated and serverless forms. With dedicated, you reserve a specific cluster configuration: you are guaranteed a given quantity of nodes, CPUs and memory, and you pay for that configuration regardless of your workload. You can scale the configuration as you need, but it won’t scale automatically (yet).
With its expansive free tier, CockroachDB’s new serverless option was really attractive for a self-funded startup like us. With CockroachDB serverless’s pay-per-use model, you are charged based on the amount of workload (Request Units) that you send to the cluster. You get 250 million free request units per month, so initially you probably pay nothing. (And if you do end up exceeding the free limit, that’s actually a good thing – it means you are serving a lot of transactional traffic, congratulations!). Your serverless cluster resources increase automatically as your workload scales, so if you are a runaway success your back end can automatically cope with the increase in demand. If demand scales more slowly, then you save money by never paying for unused capacity. It really is a best of both worlds solution.
Even more important to a small startup, though, the serverless option is as close to “zero-admin” as you can imagine. It takes only a few minutes to configure your serverless CockroachDB system, and from then on you can concentrate on building your application instead of DBA tasks.
I’ve worked with many database technologies over the last few decades, and it’s exciting to see how far database technology has come. For much of this time the overhead and total cost of ownership of running a database cluster has been massive. Today, with technologies like CockroachDB serverless, anyone can run a transactional distributed SQL database, and the cost is a fraction of what it once was (and even free when you need it most: getting started). Whatever happens with Web3 technology, I’m sure technologies like CockroachDB serverless will play a significant role in our digital future.
The gaming industry is all about delivering a great end-user experience which means building relationships with players …Read More