Tim Veil:

All right, so welcome again to Big Ideas in App Architecture. I’m your host, Tim Veil, and I am very excited today to be joined by Gregg Mojica, the CTO and co-founder of Alloy Automation. Now Gregg, you and I had a chance to catch up a few weeks ago, and have what I thought was a great conversation, and we really started by learning a little bit about what you’re doing today with Alloy Automation, what your background is.

Tim Veil:

I thought you’ve got at least in my mind, a very fascinating founding story. The timing of it all was interesting. So maybe let’s get kicked off by just tell us a little bit about you, certainly Alloy Automation, how it all came together, and then as the podcast certainly is titled, we’ll maybe spend a little bit of time talking about architecture and software and all the other stuff. But first of all, I’d love to just hear about you again and how you got here.

Gregg Mojica:

Absolutely. Thanks, Tim. Yeah, I mean, interesting story how we got here. My background, I’m originally an engineer, so wrote code way back starting in 2009, 2010, that era, got in basically when the App Store was a big thing. So I started on iOS development when all the rage was the iPhone, and this was around that time when the App Store had launched, so a lot of good stuff, a lot of exciting stuff happening at that point in time.

Gregg Mojica:

Obviously pivoted into web development, went to a work for a company called First Data, big financial services business. You can look me up on online, you’ll see that I guess I shouldn’t lie about my age, I’m relatively young still. So I ended up actually not going to college and going directly into industry, and that’s where I ended up landing at First Data, and we did a whole bunch of stuff there for a few years.

Gregg Mojica:

Everything from their developer APIs to some of their marketplace tools, you name it. I always wanted start a company. My co-founder and I were longtime friends and we thought to ourselves, we both had experience in the commerce space. Mine at First Data, which ultimately became Fiserv, and she had worked at Wish previously, so we joined forces and started to think about some of the issues in commerce.

Gregg Mojica:

We really realized that automation and integrations were a huge problem, and there’s a bunch of tools out in the space today, but what we’ve realized is that a lot of these tools are not very good for the complex requirements that a lot of commerce companies need. So we put our heads together and said, “Let’s build a prototype.” We got into YC, so this was 2019. Really this initial project was a side project at the time. We didn’t anticipate it to be a company.

Gregg Mojica:

We knew we wanted to start a company, but we weren’t sure that this would be the company. We weren’t sure how things would pan out as things typically do. So, we built this thing, launched on Product Hunt one day, got to number one of the day. We were totally shocked, blown away. We thought to ourselves, “Okay, maybe there’s something here.” So around that time, we actually had one of the founders of Webflow reach out and said, “Look, if you build this, I’ll fund it.”

Gregg Mojica:

And this was the moment in time where we started to realize, “Okay, maybe we actually have something in our hands.” So we’ve had some exciting traction. That was what drove us to apply to YC. We were very fortunate to get into YC. So we did the Winter ‘20 batch, and that was probably-

Tim Veil:

Just do me a favor, because obviously being in the startup world for a while, I’ve heard of YC, but I don’t think a lot of people… Maybe they do. I have no idea, but just for a second, I mean, talk a little bit about what YC is, because it is really fascinating, but again, I’m not sure that everybody understands the concept there or what it is.

Gregg Mojica:

Definitely. So YC is Y Combinator, and what it is is basically an infamous/famous incubator out in Silicon Valley. They’re based in Mountain View and they’ve funded basically all of the major success startups in the past decade. So things like Airbnb, Dropbox, you name it.

Gregg Mojica:

I mean, the list, there’s just hundreds of companies on this list. So getting into YC is a huge stamp of approval from the perspective of hey, they’re backing you. And then they give you this great network of other founders, and just network and advice and mentoring and so forth that really is pretty invaluable, especially when you’re starting off and you’re not necessarily sure how things are going to go.

Gregg Mojica:

So it’s basically a three-month program and they have two batches. They have one in the summer, and one in the winner.

Tim Veil:

Interesting. Okay.

Gregg Mojica:

We did the one in the winter. And it’s interesting, because you think of it like a very structured program, but it really is whatever you make of it. It’s kind of like college in that.

Tim Veil:

Interesting.

Gregg Mojica:

Well, I guess I shouldn’t be saying that though, because I didn’t go, but it’s basically whatever you make of it. It’s an unstructured program in the sense that they have programming, but you can come and go as you want. They bring in some of their previous founders, so we had the Airbnb founders talk to us.

Tim Veil:

Oh, really?

Gregg Mojica:

We had a whole bunch of other founders speak to us. Yeah, it was really neat. And then they do one-on-one mentoring and they coach you basically through building your product. The companies that are in this program are companies that they’re just getting started.

Gregg Mojica:

These are typically not companies that are established or in the market already. They have no product, maybe there’s just one or two founders and that’s it. So they’re coaching you through how do you build that first prototype, how do you get the first customer, how do you iterate, how do you raise money, and go from there.

Tim Veil:

Of course, as I’m sure you’re going to continue telling, I mean, you guys picked a… I don’t know, fortunate or unfortunate year to get things kicked off. I know we talked a little bit about that before. So I mean, 2019, we all know what comes of course later in the year in 2019, or at least what’s named for that. So yeah, you guys got in, got started, then what happened?

Gregg Mojica:

Well, what’s funny is we get into YC, and I’m super happy. I’m ecstatic. My co-founder’s ecstatic. We’re like-

Tim Veil:

Yeah. “We made it! We made it!”

Gregg Mojica:

… “This is the best thing ever! We made it!” So, we’re super excited. We get into YC and then everything’s going great. January, except we did the winter batch, so it’s in January and March-

Tim Veil:

Okay, so you even did the winter batch, you’re like right on the edge of-

Gregg Mojica:

Right on the edge.

Tim Veil:

Yeah. Okay. It’s going to be great. “We’re going to be-

Gregg Mojica:

Totally, no, we’re like-

Tim Veil:

… the next Elon Musk!”

Gregg Mojica:

We’re on top of the world.

Tim Veil:

Top of the world.

Gregg Mojica:

Literally.

Tim Veil:

The best.

Gregg Mojica:

It’s amazing. And then all of a sudden, January, February goes by, everything’s great. It’s funny because at the end of February, it’s fundraising season for Y Combinator companies is typically the last month, so you finish your product, you get some customers and then you raise some money, raise your first round.

Gregg Mojica:

So they gear you up basically in March at the end of the batch to raise money. So, in February, one of our group partners sits us down and says, “Look, guys, you want to be conservative with equity, you don’t want to raise too much money. You want to be smart, raise from the best possible investors you can get. There’s a lot of money out there. It’s a really bright future. Just make sure that you’re being smart about it and doing it the right way.”

Tim Veil:

Top of the world, Gregg. Top of the world.

Gregg Mojica:

And we took that advice. Top of the world.

Tim Veil:

Be conservative. Nothing to worry about.

Gregg Mojica:

This is fantastic. Nothing to worry about. We’ve got Y Combinator’s backing, we’re going to raise a bunch of money, we’re going to be great.

Tim Veil:

Of course.

Gregg Mojica:

Well, there’s a thing called COVID. I bet you haven’t heard of it, it’s happening overseas, starting to trickle into the US. If you remember the progression of COVID, it’s coming in of course from the East and it’s going East to West and the first thing it hits is West Coast of the United States. Seattle gets hit really hard.

Gregg Mojica:

So early the first week of March, Seattle is getting a bunch of COVID cases. People are freaking out. San Francisco I think was the second major city in the country to get COVID, and we’re in San Francisco. So now all of a sudden, people start to panic. And I remember the Mayor of SF says, “We’re putting in shelter-in-place, you can’t be in-person anymore.” So YC of course, there’s no playbook for this. YC is winging it just as much as everybody else is.

Gregg Mojica:

So we get on a Zoom call two weeks later, keep in mind it’s a Zoom call because we can’t even be in-person anymore. The same guy who told us, “Look, you got to be super conservative with equity,” says, “You got to raise as much money as you can, as fast as you can, from whomever you can.” That was the right advice at the time. Don’t get me wrong, he was giving good advice, but it was funny because I’m scratching my head going, “Didn’t you say the exact opposite just two weeks ago?”

Gregg Mojica:

That’s the startup world. You’ve got to be adaptable and you’ve got to be able to make decisions with short notice and based on the information you have, and that was our world. So we started our fundraise, we officially started the fundraise basically the day that the markets markets collapsed. I think they call it Black Monday 2020. Things were just chaos.

Gregg Mojica:

In Silicon Valley, if you have a deal, a handshake deal, that really means everything. It’s very much a word of mouth trust situation. We had investors who would never have done this before saying, “Look, I’m super sorry, but my stock portfolio went down 40, 50%. I can’t write the check anymore. Financially, it’s not a smart idea.” It was crazy. I’ll never, never forget it. It was the craziest time.

Tim Veil:

Yeah, I appreciate you telling that story because I mean, I know COVID obviously serious business, and it’s been a scary time, but I think for you and your co-founder and certainly your business, what an interesting unfortunate but interesting nonetheless time to be starting your business.

Tim Veil:

Hey, look, here we are, what is it? 2023? We’re all still here. You’re still here and you have a very successful startup. So, let’s talk a little bit more about what is Alloy Automation? I know you and I have touched on it a little bit before, and fascinating stuff. But talk a little bit more about the product and maybe talk a little bit about if you can, how you built it.

Tim Veil:

I mean, the architecture, some of those core concepts. We are on the Big Ideas in App Architecture podcast by the way, so we have to at some point talk about the tech, but yeah, just a little bit more about how you built it and what problems it’s solving.

Gregg Mojica:

Yeah, absolutely. So initially, we set out, we’re a commerce-focused company. We set out to solve problems for merchants. So we got into YC, we were initially selling the product, it was really all geared towards merchants. A lot of the store owners that want to operate, they might be operating a mid to enterprise size company and they’re saying, “Look, I’ve got a whole bunch of tools. I want to connect those together.”

Gregg Mojica:

So, we’re solving that problem for them. They don’t need to hire an engineer for that. We’ve got some great customers using the product today. Folks like Burberry and many others, Murad and so forth, that are using this. I know those are some of the larger companies, we have many small merchants as well being powered by Alloy.

Gregg Mojica:

What we’re hearing over the past several years has been that a lot of our partners, so the integrations actually that we are connecting for our merchants were saying to us, “Look, you have these integrations,” we have huge libraries. We have over 200 integrations in the commerce space today. It’s an unparalleled library for anybody in commerce. And we’re hearing this over and over and over again. So we were hearing that these businesses, these partners of ours were saying, “We want to actually leverage your technology to power our own integrations.”

Gregg Mojica:

So it got us thinking, so we actually launched another product using the same technology stack, same integrations I think we’ve built, but on top of the existing technology called Alloy Embedded. And what that does, that allows software businesses to natively embed our technology in the form of an SDK in their product.

Gregg Mojica:

So for instance, if you want to say, “Look, I want to build an integration into Shopify or Magento BigCommerce,” that might take you several weeks or months or whatever to build that in-house, you’re dealing with authentication, rate limits, and of course all the mappings and so forth with the APIs, you can do that in a matter of hours with Alloy Embedded.

Gregg Mojica:

So you just pop in our SDK, and you’re off to the races. So that’s our second offering. So we have our merchant-facing product, Alloy Automation, we have our business-facing product, Alloy Embedded, but at the core, we are all about commerce helping people to enable integrations and to save time and money from building these things in-house, where you’d normally need to outsource that to an engineer, you can do that in a low or no-code environment that’ll really save you a lot of time. And let you focus on things that are more important to your core bottom line.

Tim Veil:

We can get back into the tech, but I think one of the things that I think is so interesting given not only what you’re doing, but really the timing in which you did it is we talked a little bit about this before, COVID obviously was a super disruptive force, and not just COVID, as it turns out.

Tim Veil:

I mean, just I think the macroeconomic conditions over the last three months with the war in Ukraine and all these other things, I mean, it’s been a tough time in tech, it’s been a tough time for businesses, as they’ve had to navigate all of the changing world dynamics, economics, et cetera.

Tim Veil:

But I think for y’all, where it sounds like to me, and do correct me if I’m wrong, but certainly from previous conversations, and you hinted at it just now, I mean, driving efficiency, really. Your solution is doing things simply for people that if you didn’t exist, they’d have to go build them or build these things themselves.

Tim Veil:

So I think one of the questions I asked earlier, and I think we probably already know the answer, we can guess the answer, but how has COVID impacted the business or how have all these things impacted the business? Because one would expect that given what you’re doing, maybe it hadn’t been as painful as it could’ve been?

Gregg Mojica:

Yeah, well, it’s a great question because if you would’ve asked me in March 2020, I would’ve had a very bleak outlook and said, “The world’s ending.”

Tim Veil:

“Why did I do this?”

Gregg Mojica:

Quite literally. But it’s been interesting in the sense that think about what happened in commerce during COVID. What happened was this massive tailwind shift to focus on e-commerce. People who were historically going into a shop or boutique shop online, I’m sorry, in-person rather, they could not do that anymore. So as a result, because there were shelter-in-place orders and all these things in effect, you had to go online.

Gregg Mojica:

So these businesses also had to shift. So, there’s always been Amazon and many other folks who have been champions in online commerce.

Tim Veil:

Sure.

Gregg Mojica:

That’s not changing, but the shift that actually happened in COVID was the fact that a lot of these smaller retailers also moved online. So when they were moving online, who’s powering those integrations? Because what’s happening is when you operate an e-commerce store, you’re never just saying, “Look, I have a Shopify store,” in that sense that you have all these other ancillary applications that are powering your abandoned checkouts, your text messaging, your email flows, all these sort of things that are part of that commerce experience online are being powered by other applications. So, during that migration to e-commerce over the past few years now, Alloy was actually in a very interesting position because we were commerce-focused.

Gregg Mojica:

So, we were able to take advantage of those tailwinds. And then now of course, we have this interesting environment which has been challenging definitely for tech in the sense that there’s been a lot of ups and downs, but folks are really now of course prioritizing the products and the product features, because they’re saying, “Look, we need to pump out more features that are more helpful for our customers.” At the same time, many of those features often are integrations.

Tim Veil:

Yeah, absolutely.

Gregg Mojica:

So companies that have smaller budgets, they unfortunately don’t have the resources they had one, two years ago. They’re saying, “Look, we still need to make ends meet. We still need to drive business, and our customers are demanding integrations to NetSuite or whatever else they’re looking to do, but we don’t have the resources in-house to do it.”

Gregg Mojica:

So, that’s a really interesting position where Alloy comes in and we say, “Look, on our Embedded product, we can save you a tremendous amount of time and of course money by having us power that technology for you.” So, it’s been an interesting time.

Tim Veil:

Yeah, that’s why again, I was so looking forward to this conversation because I think just the timing of your whole story is so interesting given everything else. I mean, there aren’t too many people who exit YC literally a month before this all happens and then happen to have a startup that perhaps thrives in this thing.

Tim Veil:

Just as a quick note about Cockroach, I think we’ve found something very interesting as well. We’re obviously this database that’s multi-region and can do all these things in resilience and horizontally scalable and all this other stuff. And we found a similar thing. Here, the world is going crazy, but some businesses that were online were seeing this huge spike in traffic and growth and the technologies they were using were… I don’t know, not able to scale as easily.

Tim Veil:

So for us thankfully, although personally very challenging from a business perspective, we saw I think considerable growth over this time too. So, it’s been pretty neat. So, let’s shift if we can just to some of the architecture. I mean, what’s the stack like? What’s it built on? And then maybe if you’re comfortable sharing, in three years, did you start out down one path and realize, “Oh, whoops, architecturally, we need to maybe do something here?” Not only do we want to hear what you’ve done, but maybe where there’s some lessons learned along the way, some gotchas that you weren’t expecting?

Gregg Mojica:

100%. Oh, I would be lying if I said there were no lessons learned. So plenty of things we’ve learned over the past several years, things we’ve done right and things we’ve not done so right.

Tim Veil:

Sure, sure.

Gregg Mojica:

But at the core, so our tech is… I mean, we’re big believers in JavaScript. So the technology itself and a lot of tight script on the front end, the back end, it’s been really great obviously because we can have folks to really be full stack engineers and pivot around very quickly. That’s been fantastic for us.

Gregg Mojica:

In terms of the infra itself, so we got to get on Cockroach one of these days. We unfortunately are not at this point in time, but I think you are making a strong argument, and maybe we’ve had some calls offline about this. So one of these days, maybe that’s my answer to the next three years actually.

Tim Veil:

Yes, that’s exactly right.

Gregg Mojica:

Got it right. So in terms of our database, we’re a Mongo at the moment. We’re Mongo.

Tim Veil:

Sure.

Gregg Mojica:

And DynamoDB, so we use Dynamo for our caching because at Alloy, think about the business, we are ingesting a lot of information. There’s a tremendous amount of information coming in because we are subscribing to all these events. So for example, what I mean by that is we’re subscribing to a merchant or a customer might say, “Look, I want when a checkout is created in Shopify, do this, and this, and this.”

Gregg Mojica:

So we’re subscribing to all the events possible to make sure that we can listen in and then take the action accordingly. So, if you think about that for a second, obviously we’re ingesting this information and we need to store a temporarily caching mechanism and then send it out somewhere else. So using Dynamo for a lot of the caching, using Mongo as our core database for just the general day-to-day operations in the business.

Gregg Mojica:

We’re on AWS, so we’ve got a multi-region architecture on AWS. So we’ve got some data centers on the East Coast in North Virginia and then in Western Oregon region, hopefully international region as well coming very soon. And that is obviously because we service customers really all over the place and commerce doesn’t sleep.

Gregg Mojica:

We’re not just servicing of course the customers who purchase Alloy. We’re also servicing their end customers, their end customers being end users or merchants. And those folks could be anywhere. I mean, those folks could literally be anywhere. So we’ve got some great customers like Burberry, which of course is in the UK, we’ve got folks like Amazon, which is a massive company, is a customer of ours and they’re all over the place. So, being able to have that multi-region architecture has been tremendous for us. That’s been a big focus for us.

Gregg Mojica:

And then a lot of Kubernetes. In technology, we use a lot of Kubernetes and we are looking in the future, and going back to your three-year comment, a big focus of ours now is how do we take some of this tech and actually move it to serverless?

Tim Veil:

Interesting.

Gregg Mojica:

We’ve looked at this before in the past. It’s an interesting challenge for us because there’s serverless, obviously these are on demand resources, so these are resources that spin up and spin down as you need them, and there is a certain degree of latency involved with that, if you’re needing to warm something up quickly.

Gregg Mojica:

But we are looking at ways to possibly get around that and to move off some of our technology from just pure standards in the server-based Kubernetes infrastructure into a hybrid architecture moving forward.

Tim Veil:

It’s really interesting. So at Cockroach, we talk a lot about serverless, and I think serverless is one of those words that maybe mean different things to different people. And certainly Amazon, I did a talk on serverless database stuff a while ago, and my recollection of getting prep for that is that Amazon, I think in a lot of ways started the serverless game with Lambda.

Tim Veil:

So at least in my mind, early days, serverless meant Lambda. But I think it may very well have, and I think has evolved beyond that. For y’all, what serverless features or functions are you leveraging? Or what piece of the puzzle is it? Because it’s not probably at the persistence layer, I wouldn’t imagine, but more at the like, “Hey, I’ve got to go ingest this event or do some kind of transformation.” I mean, tell us a little bit… not me. Well, you tell me, but tell us.

Gregg Mojica:

Definitely.

Tim Veil:

All of us.

Gregg Mojica:

Yeah, 100%. Well, so two of the services that come to mind, you’re spot on obviously on Lambda. Lambda, Amazon start of the whole serverless revolution with Lambda itself, but you’ve got things like Fargate as well, that is a big service. And for those folks who don’t know much about Fargate, it’s a tool that basically lets you do on demand Kubernetes, so you’re basically spinning up pods on demand as to saying, “Look, I’m going to have a dedicated cluster and have all these resources that are already predefined.”

Gregg Mojica:

So, that’s something we’re looking into actually as well, because currently, we use EKS, the Amazon service to manage Kubernetes, but we’re looking to off-board some of our infra to that Fargate tool and also Lambda. Specifically what we’re looking at is we’re saying, “Well, how do we take some of these longer running jobs and close on our community cluster?” Because we have a mix of jobs that come into Alloy.

Gregg Mojica:

You have some jobs and this is a complex problem in the world, we’re always thinking about how to solve this. You’ve got some jobs that might be… Well, let me rephrase, here’s an example. So a customer might come to you and say, “I want to do two things. I want to sync all the historical orders from my store to my application. I’ve got 10 million orders.” It’s a lot of data.

Gregg Mojica:

And it’s not just that it’s a lot of data, but we’re bound by rate limits. So for example, if we’re pulling from Shopify, there’s a rate limit. We’ve got to retry and we got to wait and paginate. It’s a long deal. It’s not going to happen in 10 seconds. So if we’re doing 10 million orders, it might take 10 hours. Don’t quote me on that, but it could take quite a long time. So, what’s happening there is if we kill the server, that job gets killed in the middle of the job, not a good thing, so.

Tim Veil:

That’s never happened. That has never happened at Alloy Automation.

Gregg Mojica:

Never. So long story short, we’ve thought a lot about how to handle that sort of situation. And then conversely, we’ve got jobs that are real quick, “Hey, look, when an order comes in, stream that one single order to another platform or do this or that,” or whatever, something really straightforward. And that might take, I don’t know, 100 milliseconds less than that.

Gregg Mojica:

So you’ve got this polar opposite world where you’ve got a tremendous amount going on, in the sense that you could have a 10-hour, 10-day long execution. And then you’ve got a scenario where the job might run for 10 milliseconds or 100 milliseconds. So balancing that is a really interesting thing for us.

Gregg Mojica:

So we’re saying to ourselves, “Well, how do we persist some of those larger jobs on maybe our core Kubernetes infra for the long term, for those kind of jobs?” But then we off-board some of the more simpler jobs to serverless technology. Currently, that’s all running in the same queue in the sense that we have different worker pods that do designated work, but we’re looking to say how do we off-board and fragment that into using serverless? I think serverless is a tremendous opportunity for those softer, smaller running jobs where we can just turn it on, turn it off, and we’re done.

Tim Veil:

I think so too. I’m curious your thoughts on this or if this is something y’all are getting into, we’re building this serverless version of our database and I think one of the things that for us has changed is people. Our customers want to consume things in a different way than they’re used to.

Tim Veil:

I mean, EKS and Kubernetes services are a little bit like this, but this idea is you may not really have a sense of what your scale is going to be, or how much compute or storage you’re going to need. So, planning that infrastructure for the highs and lows can be difficult. I mean, I know obviously, Kubernetes can scale up and scale down, but one of the benefits of serverless it seems to us or to me is that you begin to pay a little bit more by consumption as opposed to by node or some of these other things.

Tim Veil:

So perhaps maybe a little bit more predictability and certainly, instead of having this giant thing that’s always on that maybe isn’t being used for some reason, you can scale down to almost nothing. And then okay, a job comes in, I need to execute this small batch very quickly, spin up, do my thing and then go away and I’m not paying to keep the lights on. Do y’all think about some of the efficiency there almost from a cost perspective or is that-

Gregg Mojica:

No, it’s that.

Tim Veil:

… not a big driver?

Gregg Mojica:

It’s a big driver. It’s very top of mind. You think about what we do, different times of the year or different times when folks are buying things, like Black Friday obviously, you’re going to have a lot more traffic on Black Friday than you’re going to have in the middle of June one day. Because for commerce, Black Friday is a big time, the holiday season, another big time.

Gregg Mojica:

So for us to be able to quickly scale up based on the demand is tremendous. I mean, serverless, it’s billed as infinitely scalable. So you’re really not worrying about, “Gosh, am I going to have to set up this monitoring system and worry about the cluster having a problem at certain capacity?”

Gregg Mojica:

Obviously those things are problems we’ve solved and we worked around the limitations of a non-serverless architecture. But as we add more and more users and we encounter more and more complex problems, I think that the scalability and also the cost is fantastic. We’re in a recession, whatever the government says. I think we’re in a recession. So everyone’s thinking about cost-cutting and being efficient with capital.

Tim Veil:

If they say it, it must be true. Right?

Gregg Mojica:

Absolutely.

Tim Veil:

Because I know we’ve talked a little bit about it, you’ve mentioned you’re building on top of it, Kubernetes, and I wanted to get you in trouble here, but Kubernetes is one, maybe not as dramatically so as other technology topics, but there to me seems to be a little bit of religious fervor here. People either love it and it’s God’s gift to everything.

Tim Veil:

Some people I think are starting to maybe tire of it. I mean, what’s been your journey? Kubernetes is not easy, and I think you have to have quite a bit of skill internally or skill you can rely on externally to help manage. What’s been your journey with Kubernetes, just out of curiosity? I mean, is it something y’all have had some great resources that knew it well, has it been a learning journey? I’m just curious, what’s your overall experience and then thinking on Kubernetes as a tech is?

Gregg Mojica:

No, I think we’re biased, I mean, in the sense that we started on Kubernetes. So I think had we not started on Kubernetes, I might have a very different response to you, in the sense that if we’re migrating or God knows what, there’s a lot of work involved in that, but we made an intentional incision in the beginning to say, “Look, we want to make the most scalable infrastructure as possible,” and Kubernetes was a great solution for that.

Gregg Mojica:

There’s certainly a learning curve. I mean, there absolutely is. But how we’ve architected our system is we’ve got the core API running in one group. We’ve got what we call worker pods and those pods are running another group and that is handling the day-to-day jobs. We have another group entirely called our long-running queue, and that’s what handles those many possibly 10-day or 10-hour jobs.

Gregg Mojica:

So we’ve been able to take advantage of that substantially because it’s just big baked in. It’s really easy to spin those up versus setting up an infrastructure around that, building it yourself, it’s very difficult. So we’ve been able to just use this tech and for our business, it’s been really great. I think it depends on those type of business you’re operating.

Gregg Mojica:

I mean, if you’re a more standard web application where you’re not maybe doing job orchestration and really complex stuff like we’re up to, you probably don’t need it. I mean, it’s probably overkill. You can probably honestly be really effective with a serverless tool like Lambda and only rely on that or have an EC2 instance that does something great for you, but our infra is a little bit more complex, so we’ve needed to-

Tim Veil:

Yeah, it’s just so funny because in my role as head of solutions engineering, sales engineering, and I get to talk with companies all over the place, and big and small startups, very large enterprises, and it’s funny sometimes. You talk to folks who just get enamored with the technology, because they’ve heard about it and they want to bring it into their organization or their CIO or CTO or CEO says, “Hey, I heard of this thing, and we got to push this thing down.”

Tim Veil:

What we end up encountering is, and particularly with Kubernetes I think because we’re a database and you can run it on top of Kubernetes, but we use StatefulSets and it’s a Stateful service, which is not traditional Kubernetes, but you end up getting folks who are like, “Hey, we’re going to run your database on Kubernetes.”

Tim Veil:

“Oh, that’s awesome. Do you have a lot of experience with Kubernetes?” “No, no, not too much.” And it’s like, “Well, wait a minute.” Learning a new database technology is one thing. But trying to combine that with something I think as complicated as Kubernetes, that might be a rather tall, tall order.

Gregg Mojica:

Right.

Tim Veil:

One other thing I know you and I talked about before and just curious, you’re coming to us live from Hawaii, and I know when we talked previously, I think you guys are using regions in the West Coast of the US and West Coast to East. Latency’s a thing. I mean, the speed of light is a thing. We can only get your packets so far so fast.

Tim Veil:

I know you briefly mentioned thinking about maybe expanding into European data centers. I mean, obviously given Cockroach, we like to talk about multi-region and resiliency and latency and all those things. I mean, how have y’all managed your infrastructure, latency customer experience knowing that they’re in different places and depending how far you are from those places, different things can happen. Is that something y’all think about, or is that just what Cockroach people think about?

Gregg Mojica:

No, no. We think about it. I mean, I was in Dubai last year. It was funny. So I was in Dubai last year. We had actually an off-site there, and we did not have a multi-regional architecture by then. And I remember sitting there and we were working, we had a bunch of folks there and I was going, “Gosh guys, our site is so slow.”

Gregg Mojica:

And they’re like, “Yeah, it’s because you’re running out of North Virginia.” I was like, “Oh, geez. Okay, okay.” So it’s definitely top of mind. I’m in Hawaii for the winter, snowbird here, so very happy we’ve got nice weather over here. But Hawaii is the middle of nowhere. I mean, Hawaii, Hawaii’s a funny situation because they’re not a data center in Hawaii.

Gregg Mojica:

So the closest thing to us is we’re processing through Oregon, which is the closest location I think geographically, or California, one of these data centers. But there’s nothing really closer, but if you’re a customer anywhere in the contiguous United States, having a close data center to you is really, really important for latency.

Gregg Mojica:

And it’s very, very top of mind for enterprise customers. Example of this is Amazon, we work with Amazon and for them, they need it. I mean, they have to have a certain degree of latency as part of an SLA. So we are very intentional on that, and going back to actually the serverless component as well, that’s another great reason to use serverless, because serverless I believe actually has this multi-regional architecture baked in.

Gregg Mojica:

So when you fire a Lambda function, it’s triggering off whatever is the closest data center to you. You’re not even worrying about that, which is really, really neat. If you’re using a more traditional architecture like ourselves, you’re using a tool like Amazon Global Accelerator, which will basically proxy that information over to the nearest data center to you.

Gregg Mojica:

So, if I’m in Hawaii for instance, it’s going to route that information over to for example, our Oregon region. If I’m in let’s say Kansas or I shouldn’t use that, Boston for instance is going to route us over to the East Coast location. So it intelligently knows that, but that again is more intentional. We are thinking about that, and that is something that is top of mind for us. But I think Lambda and many of these serverless tools have it baked in. So, it’s a tremendous reason for us to leverage that further.

Tim Veil:

It’s interesting. So, we had our revenue kickoff, revenue team kickoff this year and our CEO Spencer came down and mentioned something which maybe is a thing we’ll start to hear in the world, or maybe it’s already out there, but this idea of almost regionless, and I think in a lot of ways, serverless is that way. I mean, it’s no longer about which regions are you in, you don’t even have to think about regions really anymore when you get to a certain level, and maybe you shouldn’t be thinking about them.

Tim Veil:

I just have some endpoint. I know it’s always going to be up. I know it’s always going to serve me correct data, it’s going to be fast, and that could be anywhere. I don’t have to consume myself with worrying about if the lights are still on in Reston, Virginia, or wherever that data center is in Oregon. That’s awesome stuff.

Tim Veil:

I mean, have you been in situations where, and do you test for, “Hey, this region’s out of service?” Or are you routinely taking regions out of service in order to do stack updates? What’s been your experience there? And it may not be something everybody does, maybe y’all aren’t doing it. I’m just curious if that’s something that comes into play.

Gregg Mojica:

It’s something we’re looking at. I mean, how our system capability works is, so how we’ve addressed some of the actually concerns I mentioned before about what happens when a job gets shut down in the middle of an update, how we handle that is essentially, we will actually notify our pods. We’ll say, “Look, you’re running, for example, a long running job.”

Tim Veil:

Heck yeah.

Gregg Mojica:

We know this is going to take… You’re looking to grab 10 million records, it might take 10 hours. We put that on dedicated infra. That’s what I was saying before-

Tim Veil:

Interesting.

Gregg Mojica:

… about that other set of pods that we have. So we’ll [inaudible] other set of pods and then we’ll notify that set of pods and say, “Hey look, you have nine days to finish these jobs. Go figure it out.”

Tim Veil:

Interesting.

Gregg Mojica:

And if you don’t figure it out, there’s an update coming, you’re going to get shut down. And part of our SLAs and our business facing documentation and so forth is we’re telling folks these things are not going to run infinitely, because they can’t. We can’t run a job for you for three days, or-

Tim Veil:

It costs you a little bit of money if you did, too.

Gregg Mojica:

Exactly. And also other customers would be affected. So what we’re saying is, “Look, there is a time limit,” but we were going to power it down. And then for instance, for maybe the shorter jobs, the things that are just when the order comes in, do this, whatever, we give it a much shorter number. I think it’s like five minutes or 10 minutes, something reasonable offhand say, “Look, hey, guys, here’s the jobs.” Just know when I say, “Hey, guys,” I mean the pods.

Gregg Mojica:

“Hey pods, just so you know, we have this new update coming in, start powering off,” so they finish their jobs. And then we power that off gracefully and that’s replaced by a brand new pod that has the updated code base. So what’s nice about that is we don’t really necessarily need to worry about okay, we got to turn off North Virginia and proxy it through Oregon.

Tim Veil:

Yeah, yeah. that’s what I was wondering.

Gregg Mojica:

And turn Oregon back on after it’s done. Yeah, actually that’s built in Kubernetes. That’s another reason we like Kubernetes, but I mean, I’m sure our thinking will continue to evolve as we encounter more challenging situations.

Tim Veil:

Well, I’m sure just like I have, I mean, it’s amazing. You think one thing is the right way to do it, and then a couple years later, who knows where the-

Gregg Mojica:

Always a better way, right?

Tim Veil:

Doesn’t it seem that way? There’s always some new thing on the horizon, some new, better way. Well, maybe that’s a good segue. I know we’re running up on time and you’ve been very, very generous to spend this amount of time with us. Maybe if you don’t mind, leave us with final thought about what you’re looking forward to this year, whether it’s personally or professionally at Alloy, just would love to hear what you’re excited about going into 2023 here.

Gregg Mojica:

Well, I’m always excited about our tech. Our tech, I think really it’s a challenging problem and I think I’m an engineer by trade. I like solving challenging problems and our team loves solving challenge problems. So, that’s always a big thing for us. But what I’m really excited about honestly is hiring more people.

Gregg Mojica:

I mean, we’re growing the team. We’re in an interesting position in the sense that we were able to raise money fortunately actually before the economy started to collapse last year. So we’re in a good position, and we’re still hiring, many, many open roles. And it’s a good time for us in the sense that unfortunately, a lot of other companies are laying off, but we’re able to have the opportunity to hire more.

Gregg Mojica:

So, that’s really been fantastic. So I’m excited for the team, get more folks, get more talented folks on the team, and the company and everybody, we all level up together. So, it’s a win-win situation for everybody and super thrilled about that.

Tim Veil:

Well, Gregg, that was awesome. I really, really, really enjoyed the conversation, learned so much.

Gregg Mojica:

Of course.

Tim Veil:

You’ve got such, like I said at the beginning, I think it’s such a fascinating founding story. You guys are building great tech at the right time. So really, really-

Gregg Mojica:

Appreciate it.

Tim Veil:

… enjoyed spending time together, and hopefully we’ll get you to consider Cockroach sometime in the near future, but we won’t push that hard.

Gregg Mojica:

Come on. That’s on our next three years roadmap, like we just discussed. You got me.

Tim Veil:

That’s all we ask. Just get it on the roadmap. All right, Gregg, thanks so much.

Gregg Mojica:

Thanks, Tim.

Tim Veil:

Thanks again for joining us on the Big Ideas in App Architecture podcast. Wherever you’re listening to this, take a moment, leave us a review, give us five stars, tell us what you think. It would be certainly much appreciated. Look forward to seeing you again every Tuesday.