Jason Valentino:

If anyone does listen to this, and if you’re listening to this and you’re working in an innovation zone, you’ve probably heard, “Well, how’s that going to scale for the greater company?” And there’s no more fun way to prove people wrong than just take over the biggest thing they have that they’re worried about.

David Joy:

What is up everyone, and thanks for tuning in. In today’s episode of Big Ideas in App Architecture podcast, we speak to Jason Valentino, who is currently the head of engineering enablement at BNY Mellon. Jason and I talk about his experience building innovation pilots at Cockroach Labs to how the ideas were incubated and led to amazing products that got shipped into production. We also talk about his experience building developer experience teams, and an amazing platform engineering practice at Peloton. So pump up that volume and get ready for an insightful conversation with Jason Valentino. 

Welcome to the podcast, Jason. How are you doing?

Jason Valentino:

David. Doing well. How are you, my friend?

David Joy:

I’m good. I’m good. Here is one of the first things I wanted to say. Jason Valentino is a really, really cool name. I don’t know if anybody has complimented you before on that.

Jason Valentino:

Thanks. No. I am blessed with a great last name and can’t speak Italian at all, which is quite upsetting to go along with it. I’m actually Jason Dante Albert Valentino.

David Joy:

Oh, wow. Okay.

Jason Valentino:

I’m as Americanized as you possibly could get with a name like that.

David Joy:

You know what the name reminds me of? If you have a band like a nineties or I would say seventies, eighties band, and then if you say you are a main vocalist for that band, that’s a fitting name actually, Jason Valentino opening for this band or something like that.

Jason Valentino:

I’ll take a note of that. I’ve been thinking about what to do next in my career, so maybe that’s where we go with it.

David Joy:

Awesome. Well, I would say first of all, it’s so great to have you on the podcast. I’ve heard so many great things about you and I’ve been following some of the things that you’ve been doing on LinkedIn. It’s an absolute pleasure to have you. And right now I think what your current role is, you’re the head of engineering enablement at BNY Mellon, which is a new gig that you just started. Tell us a little bit more about where you are right now and what led you to that opportunity.

Jason Valentino:

Well, I just started, I actually just completed my first month at BNY. It’s a slightly new role for the organization where like most companies, there’s a team that is managing dev tooling. There’s teams that are doing automation, cloud and whatnot, and there’s a huge desire for reuse throughout the company. And they’ve really been debating, how do you actually measure developer effectiveness and happiness? And it’s just one of those things where I think the internal debates turned on to a point where they’re like, we should probably find someone that’s done this before. And then my manager, Mike Keslar, somehow found me through the DevX network.

And so it’s just one of those unique situations where I’m talking to the highest levels of leadership in the company and they absolutely have a passion for this. Whereas normal companies, when you are the DevX person, you’re always having to sell up. Always. You’re like, “Hey, here’s the value we can get. Just do this X, Y, and Z.” And so just the idea of having a company where I’m starting with tailwinds behind me is absolutely, absolutely exciting. Now, I did promise myself I’d never go back into banking after working in banking for a dozen years. We all lie to ourselves, so it’s fine. It’s fine. I didn’t break any core tenets here.

David Joy:

That and as long as a paycheck is good, right?

Jason Valentino:

Yeah, there you go.

David Joy:

You’ve had a really amazing, fascinating career, almost 20 plus years of career, but the last 10 years of your career has been really fascinating. You were the number third employee at Capital Labs that produced some amazing results. Capital One.

Jason Valentino:

On the tech side, the product grew quicker than tech did, but it was probably number three in Capital One’s… Now what they consider their Capital One labs or the Innovation Lab back in the day ran most of their infrastructure functions. And so it was like, this is so long ago, but over a decade ago, the bank’s policy on things like cloud or big data was, “No, don’t touch it. Don’t talk to vendors who touch it.” And we ran the first set of exceptions around those, experimented with the cloud, inevitably sat there in front of Cap One CEO and showed off the technology that years later is now the only way to get anything done in the company. I think they shut down their last data center maybe four or five years back.

David Joy:

We’ll dive into that bit a little bit more, but in your career, the next step was a really good stint at Peloton as well at the company that we all have known about since COVID. And you were there right at when everything was going wild at Peloton with all the subscribers and people buying Peloton during COVID. But you’ve had a fascinating career when I was going back and looking at what you’ve done. You’ve been especially part of enterprises like Capital One, Peloton and now at BNY talking about cloud enablement developer experience and just helping the companies build these early prototypes that have turned into amazing products for them. That’s what I was really curious to talk to you about. Before we get to BNY and what you do, I wanted to start with just talk about the piece that you were talking about. How did that Capital One gig come through and what made you excited to get into that?

Jason Valentino:

It kind of chose me. Before that, I was an IBM consultant at Capital One working in the data center, trying to coordinate all the hundreds of things that need to go into building infrastructure at a company that loves its requirements. And at the time was still really waterfall. And I guess I was the one who cheated the most at that job. My team had the fastest numbers. I became a full-timer at Cap One, and it was because we cheated. We pre-baked designs, we knew the things the architects were going to ask. We just tried to… We escalated our stuff through.

David Joy:

Got you.

Jason Valentino:

But in the old days, getting stuff done and it was hard. And so you had to cheat. And when the gentleman Skip Potter, who now is the CDO CTO at Columbia came in to start the lab, he was like, “Well, you seem like the only person that’s getting stuff done here. Why don’t you come work for me?” And so it went from me just being an average engineering manager in the company to something that really springboarded my career. It was exposure in the lab, this is yet again a while ago, to concepts that you never really get in tech, like design thinking and really having some empathy to your customers. And for someone who was in infrastructure at the time, my customers of the dev is trying to get work done. And so we ended up rolling a whole CICD suite out for the lab and building tools specifically for the labs so the lab could move quickly.

And then as Cap One matured, similar practices or exact same practices were adopted by the greater company. And so Cap One actually became an easy place for all devs to work to a certain extent. And once I had my first couple of projects that work out in the lab, I rode one out and ended up getting into just… I went from having the smallest thing in the company, which is all these lab pilots, to I got to own most of the Orchestrator and SRE operations behind everything that was customer facing at the company, which was super cool.

David Joy:

Nice.

Jason Valentino:

It was like you always hear when you’re… If anyone does listen to this, and if you’re listening to this and you’re working in an innovation zone, you’ve probably heard, “Well, how’s that going to scale for the greater company?” And there’s no more fun way to prove people wrong than just take over the biggest thing they have that they’re worried about and start applying practice to it.

David Joy:

That’s awesome. Last time when I was talking to you, you told me you were one of the biggest troublemakers at Capital One because you were one of the first people to test out the cloud and bring that in. And now in hindsight, and you can attest to this, Capital One is one of the biggest users of AWS. Tell me a little bit about that part as to what led you to believe that okay, this is the direction the company needs to go. Cloud is important.

Jason Valentino:

It wasn’t that I was a good strategist, especially back then. It wasn’t that I was a good futurist. It was just where’s the resistance and where’s the tension? And when you’re trying to manage your data center as a company, that’s probably not your specialty. Nothing feels fluid, everything feels slow, everything is baked in process. And also these are… A lot of data center management is, there’s a lot of just, here’s the overhead behind this, here’s the project management behind this. This is the capacity planning behind this. It’s hard to do, so it’s a lot easier just to open a little Terraform file and push go and have your infrastructure done for you. From my perspective, we just started looking at things that were easier and faster. And as time went by, the rest of Cap One saw the same. I was all pilot’s, little stuff lavvy versus once the value of the cloud was seen, some of the top execs in the company and some of these really high performing teams started rallying around the cloud.

Next thing you know, they’re encouraging every new project to go there instead of sitting in the data center. And I was fortunate enough, because it’s one thing to have cloud capabilities, it’s another thing to have a team that knows what to do with them. I was fortunate enough at the time that I had a pretty software shaped team that was ready, willing and excited about it and able to start getting stuff there. One of the other execs in the company and I had a joke like, which one of us can get our biggest platform to the cloud first. It was just a gentleman’s bet over who probably more recklessly throw something into the cloud, but we got the mobile app in there quick, Connects and all the other customer facing apps really quickly followed.

David Joy:

I wanted to ask, what was the culture before you started exploring that. Was Capital One mostly… I know Capital One went through a really big moment where they were like, we need to be the leading innovator in the banking space. And they were not really a bank bank, but more of a credit card approach, and then it suddenly grew up. When you joined the company, what was their perspective and what did you feel they needed to do in term to move into that innovation space?

Jason Valentino:

Oh, now looking back, what I thought they should do?

David Joy:

Yeah.

Jason Valentino:

Because, I didn’t at the time. I joined in 2009 I think, and right around that same time period, maybe a little bit before, I think their current CIO took the helm. I think Rob should probably get all the credit for what… A, you asked about the culture. The culture was already strong. It was a young company, it’s founder led. If anyone’s ever worked there, it’s fun to work there, even if you’re staring down obstacles and change controls and weird stuff. It had all the bones of greatness. Rob walked in and honestly he saw outsourced development and did that. He saw a lack of bringing in our own talent. He started a college hire program, he started a lab, he did this, he did that. And so the building blocks were all going into place right around the time I was joining and what followed, once you start really intersourcing and actually having an innovation agenda, moving away from contract development, moving away from other organizations, managing your data centers and other important aspects of your company, good things tend to follow.

And that combined with that founder led nature, that Cap One had, the next 10 years, I think anyone who was there during that journey, it was history. This thing went from old-fashioned bank to cloud first business as quickly as I think anyone could go.

David Joy:

Well, that’s awesome. Tell us a little bit more about one of the incubation projects that you started off in the lab and that turned into a big important significant product for Capital One. And then how did that… Take one example and talk us through the journey of looking at the product, thinking about design, architecture, scale, all those things, how you considered all of that.

Jason Valentino:

Put the infrastructure guy on the spot for speaking to-

David Joy:

No, we’ll not go into details, but you can tell us the high level.

Jason Valentino:

I did. One of the fun ones we were doing way back in the day. If you can remember, and this is a long time ago, remember when the iPhone six came out?

David Joy:

Yep.

Jason Valentino:

iPhone six was a really big milestone for the Apple products, and I’ll tell you how this ties into the bank at all, but it also was the iPhone that had Apple Pay. And so most of the big banks got invited secretly into the conversation with Apple around how can we be a launch partner with Apple Pay? And the lab along with partners in digital all had to figure out exactly how we were going to get a companion app up running in a really short period of time for this launch so that our customers have it, can easily just pull up one of their Cap One cards, take a picture of it and have it auto-enroll into Apple Bay. And so that was a big partnership between the lab, greater architecture, greater digital engineering organizations, and that was the ride out for me. Once we got that… I ended up being the infrastructure head on that project and once it was up and rolling, it was time for that to graduate out of the lab and be run by a bigger organization. The healthy move to make was to go with the product.

David Joy:

Got it. That’s amazing. Was that also… When you were creating the product, when you were designing the whole idea of how it all comes together, you have to definitely consider transactions and the scale with these transactions happen, you have to consider compute. How did you go about recommending that? At that point, did you have a set of architects who were educated enough because you also had the function of developer experience that you were slowly honing and bringing into the overall culture at Capital One? How did you do that?

Jason Valentino:

I wasn’t doing any DevX work at Cap One, especially back then.

David Joy:

Not at Cap One-

Jason Valentino:

I was just managing a couple tools.

David Joy:

Okay.

Jason Valentino:

But at the time… Yeah, there was a lot of that. Most of the difficult part of the flow when Apple Pay first came out with more enrollment-

David Joy:

Got it.

Jason Valentino:

… than actual transaction loads. Cap One knows how to manage transactions.

David Joy:

Got it.

Jason Valentino:

There are plenty of credit card swipes having over the course of a general day for that. I think they see something like eight, 9% of all credit card transactions in the world.

David Joy:

That’s why.

Jason Valentino:

Not saying scale isn’t something that you plan out in a project like that, but for us and the infrastructure that I was working on specifically was enrollment.

David Joy:

Now, let’s take a little bit of a pivot back and maybe go back in time a little bit to a young Jason Valentino and understand a little bit about where were you in terms of when you were finishing college and thinking about, “Hey, I need to get into tech,” and what motivated you to get into this field of tech 20 years ago? Was there a trigger point, something that you saw that really excited you about the field?

Jason Valentino:

No, it was just the ability to create. And so I had my own little mini business towards the end of high school and into college making websites for local businesses. Now tell everybody how my age is, it was the late nineties, and if you could figure out that marquee tab, you were basically the greatest HTML programmer in the world at that period of time, so let’s not get too impressed. But did a little web dev that was all self-taught way back in the day, which was super fun. And then as I was in school having that actually just randomly met someone. I taught karate. I thought I’d be a professional kickboxer when I was a kid and here we are not doing that, but someone at the school was like, “I managed a tech team. Do you want to be my intern?” And so it went from… My last dev gig was before I ever even got through college.

And then next thing I know I’m working back office at some weird pharmaceutical company imaging computers and keeping servers running. And that just within six months I was a full-time engineer on the staff and kept rolling from there.

David Joy:

Nice. That’s awesome. And while you were doing that, did you ever anticipate that you’re going to be a head of engineering enablement somewhere down the line? What were you thinking at that point of time, like a dev career? Is that where you were lying?

Jason Valentino:

I honestly, and this is bad advice… This is so counterintuitive to people that actually are really goal motivated or have really great growth mindsets. I’ve never really considered my end game ever being further along than what I’ve done every four years of my career. It’s just like I accidentally keep leapfrogging where I think the ceiling is and I have no clue why or how.

David Joy:

That’s funny. But it’s interesting when you say that because I’ve seen a lot of people I’ve spoken to in my life who always feel like having a time period set for what they want to work on and accomplish. It’s just way more simpler and much more, I would say, objective than saying, “Oh, in 20 years I want to be a CEO,” when maybe you can become a CEO after four iterations of objective completions. You feel like you want to start a company and you’ve acquired enough knowledge. I feel like that’s a great advice also generally for people to be aware of, hey, think short, be clear about what you want to accomplish in that time, time period. What do you think?

Jason Valentino:

I don’t know if it’s advice, it just worked for me. For me, it’s just like, “Hey, if I do this for three years, what do I want to work on next?” If you ever had to read the terrible word vomit, that is my resume. I’ve bounced between EM and SRE and EM and SRE or whatever the time period equivalent are over and over and over again to the point where it’s like they’re both fun. You can give really good at one and the other and have a great understanding of what needs to be done to keep software running. But it only happens if you keep volunteering to bounce back and forth. Before Peloton, I was the defacto CTO of Cap One Shopping, running the full stack, learning way more about front end than I ever thought I would.

David Joy:

When you were saying you’re talking about EM and SRE, that’s interesting because when I was talking to you, when you were saying that, I realized that SREs have had a moment in the last decade, especially with infrastructure and Kubernetes projects and container orchestration being so central to company’s scale and growth. And you’ve been a part of that growth, at least at Capital One Shopping. You were talking about how, when we spoke last time, how containers became really quickly important for you. Tell us a little bit more about thinking like an SRE while also thinking about building a scalable product.

Jason Valentino:

And it was previous to shopping the gig right before that was when I ran digital reliability. And the problem statement there, which most folks would listen to now, and it’s pretty obvious and easy, but this is six, seven years, maybe eight years back. You get to the cloud, you have a well-designed app to run on smaller increments. You have microservices, micro experiences, but you have 150 to 200 of them and you’re just one app. You’re not the whole company’s ecosystem and this is pre-containers and you’re at a company that really insists you need this many availability zones and this many live regions. Next thing you know, your AWS bill is more than you’re ever going to make an income in your life, so you got to do something about that. And for us, that necessitated moving to container-based delivery, which we did.

We built a custom hosting app using Hashi’s Nomad at the time. Kubernetes wasn’t right for us just yet, and the company has really made strides from what I can tell since I’ve left there. But then also you also just consider in your architecture for us in digital reliability, those services, most of those called back to something else deeper down in the depths of the bank. And so you never really think about it, but if whoever your retail bank is, if you look at that mobile app, there’s your profile picture, there’s your bank balance, there’s your credit card balance, there’s something else there. There’s a bunch of other services, and those aren’t common from the team that manages that mobile app. They’re coming from somewhere deep down in the company and I think just as many services as we had, we had that many or more dependencies on other folks.

And so when you get to a point where you’re delivering content on behalf of the rest of the organization, you got to play it like Netflix. Easy thing to do is just steal their stack, borrow open source, cool, but get to a point where you’re using defensive programming. You have circuit breaker set up for everything behind you. You have quick ways of filtering and having request filtering if something evil gets out into that mobile app. But you have to get to the point where if someone else in your company is having a bad day, the customer doesn’t know about it. Your portion of the application can absorb it. That being the case, it was really, really easy to say, “Hey, look at my uptime, my uptime’s great. Don’t ask me about every component in every backend dependency you’ve ever had.” If you have just a 100 dependencies, your uptime metric is like 99.99. Someone’s going to break every couple days, and that’s to be expected, design around it.

David Joy:

Well, that’s amazing. And I wanted to get your thoughts too, because I’ve thought about this often. And off late, I’ve had a few conversations with really good SRE folks, and it also feels like you have some amazing experience there. 15 years ago, I would go to a website and the website wasn’t hitting. I was like, okay, I’ll just go back and tell somebody from my crew, “Hey, this is not working. Okay, we’ll wait to book the ticket for the next concert when the website comes back up.” But today, you cannot do that. Today, something goes down, immediately you have people throwing a tantrum on Twitter or now call it Instagram, Threads or wherever it is, and everybody knows immediately and hits the reputation of the company, hits the customer experience. Tell me about how much, when you’re designing systems, you have to consider these kind of situations.

Jason Valentino:

When it comes to looking at stuff like customer impacts, error budgets, whatnot. For me, it’s more I just relive every incident retro I’ve ever had in my head. And I try to think through because how many… In the old days you used to sit at them and someone would be like, “How are we not catch this in testing? Or how could you put this problem in there?” And you blame the people, instead of just recognizing humans are flawed. I’m a super flawed human, and so I commiserate with this. How do we make the software a little bit more elastic to actually allow for a couple human errors here and there? At Peloton, we had a pretty significant outage, and the one thing I really respected about Peloton is they were really good… The IM practices they put together were quite good and quite blameless.

And so it was very easy to step out of a public outage and go, “Wow, okay. I do want a paper on how that happened, why that happened. I also want recommendations from the engineering community on if something similar were to happen, how could we survive it without this being a problem that exacerbates some other underlying thing that’s designed poorly that fails the company at the end of the day?” Long story short, assume we’re all going to… Assume we all are going to mess up and try to design your software as such.

David Joy:

It’s funny. It reminded me what you said, was the first job I got ever tech job was in a team where I had to do development testing and support, but within that team, I had to do support first. I was the youngest guy, and this is my first day and I was working for a consulting company at the time, which was working for British Telecom. And British Telecom had a solution called ADSL. Basically, what you could do is you could go and put in your postcode or a zip code and it would tell you what bandwidth is available? And other retail companies would buy that to sell different broadband solutions back in the UK. This is the first day of job. I’m doing the evening shift, which is like seven o’clock in the evening, and I have one guy with me, they said it’s a P one because apparently the entire website went down.

When you say SREs, I was like, that moment I was like, “Dude, I don’t know what to do here.” But then we went through the process, look at different tickets. And back in the day what people would do as well, it seems like it’s upstream teams issue, so we would send tickets to each other, play table tennis over there. But we eventually found the issue was that somebody, when they checked in the code in the last release, forgot to add a function and that was really important, that would check a certain variable. And then that was the reason why it was erroring out, and then the whole website would go down because of that. We have gone way past that in today’s system because we are building not monolithic application, but microservices and we release of microservices API or different function than they run, now run in containers, maybe on an EKS system or a GKE, and then the maturity has been fantastic.

How do you feel about where we have come in the last 15 years? It’s been fantastic, and especially for where you are, it would be a very good perspective to see all of this.

Jason Valentino:

It is, and just the best part about it’s when Gene Kim first wrote The Phoenix Project, you could read it and commiserate with it and it told you what was wrong, but you looked at your own team and you’re like, it’s going to take a while before this is right. And then you speed up to God knows how many books later to the modern day, the average engineering manager cares about their uptime, cares about the stability of their system. Just getting to a point where we’ve moved away from a world where there’s Ops and there’s development and we’re all incentivized differently and we all have different opinions on what we need to do next. It’s just getting to a point where the teams now have autonomy that actually, “Yeah, I care about the stability of my application.” This is, not only is stability a first class citizen, but so is design, so is just the overall architecture of mine. I think just getting to a point where we all have more ownership over everything was probably the biggest win I’ve seen in the last decade.

David Joy:

No, that’s awesome. I was going to ask you, you went from, of course, you’re an engineering manager, so looking at engineering enablement, you’ve had great SRE experience, but you’ve also been a great leader. You’ve built lots of different teams. What’s your leadership style? How do you inculcate different ideas on how teams need to build or follow actively on what the objectives are and things associated with the goals?

Jason Valentino:

I don’t know where you’ve heard that I was a good leader, but okay, we’ll work with it. We’ll go on the operating side.

David Joy:

I generally give everyone the benefit of the doubt and say they were good at everything.

Jason Valentino:

Fantastic. Okay. For me, it’s a highly effective team is one that can move quickly without my input or without my leader’s inputs. And so I tend to be pretty… My goal is to be hands off. My goal is to let the team’s charter and the team’s operating principles work for themselves. My job is more help folks get what they know they need from the greater organization in order to get their jobs done, like run people through exercises they haven’t. An example of that is, I have a team that feels like it’s understaffed right now. Objectively, they’re probably understaffed based on what I’ve seen for similar teams. But how do you display that information in a way that you’re going to get buy-in from others? With that team, it’s sitting there and going, “Here’s how I would tell the story.” I would show all the things you do, show what the priority one… Declare what is priority one for all of these different apps that you support and show how much staff goes into that important part.

And then somehow show where product intent falls on that list. It’s A, teaching people the tools that I developed along the way. B, making sure we have a really strong charter. I don’t think of a charter as a check the box, “Hey, I need to have this document.” I see it as something that actually teaches the people on the team what our priorities are, how to scale… How to actually assess new priorities, what our mission is, how I’d like them approaching decisions if I’m not in the room, because I don’t want to be in the room. A strong team is one that can at the lowest level of the team, make the same call as the person at the top of it. And I probably do some other stuff too. I don’t know, call somebody that has worked for me and just find out what-

David Joy:

After this I’m going to call one of your employees or somebody who worked for you. But I think what you said is very interesting. Leadership is, sometimes you have to step in and inspire, but sometimes you also have to let the teams figure out how do you want to communicate upwards? They need to learn that themselves. And have you had instances where you are trying to make a decision on what product to choose and there’s really been a confusion? And then you have to step in and say, “Okay, we need to look at these fundamental objectives before deciding, and we’ve had a challenge like that before.”

Jason Valentino:

Sure. To me, I think it’s more, it’s not a step in when the decision is trying to be made. It’s more of a start with guiding principles. When approaching a big, big project, it’s the, what is my job? Is my job to tell everyone to go design something and then at the end of it, shit all over the design. Or is it to say… And I don’t mean requirements. I mean like, hey, I want to actually paint the guiding principle story, why we’re doing this, what’s the history here? I want to feel, people who may not be familiar with the project along the way. I want this to be very consumable story. Maybe the company’s tried this three times and this is the fourth time and here’s what went wrong. And then go into it, say, “Hey, for me to be happy as the sponsor boss, whatever my name is today, here’s what needs to be true at the end of this.” And then share that with the world.

Share it with the world and get the world to also chime in because I want to hear from the other leaders and then their folks, what would make this project successful. And then if I hand that to my team and I say, now you’re in design phase, they’re going to make something great. The expectation’s already been set. Everyone’s already chimed in on what they generally want. Sure, there’s some versions of it. And this to me is all internal customers in this particular example. But there there’s going to be some comments on how we design it along the way, but the expectations already set that we are going to design. We are going to build something that does do this. And so for me, it’s just one of those…. I think my job is to listen to people, regurgitate what I think they told me as far as what they want my team to work on.

Then somehow come back and say, “Hey, my team has designed this thing. Could you read this and make sure it’s the thing that you said that you wanted me to work on?” And then as we build it, say demo the thing and say, “Hey, is this still the thing you asked for and said it was the most important thing.” It’s just like that. If you can create that quick feedback loop over and over and over again, especially someone who works in engineering tools, by the time you roll it out, everyone’s like, “Ah, it’s here. It’s fantastic.” Instead of the, “Hey, we think we know how to make your problem going away.” And then just disappearing. And a year later coming back with something that everyone forgot about and no one really has their eyes on anymore.

David Joy:

No, I think when you said that, I was exactly thinking the same word, that what you’re creating is a very good efficient feedback loop. Before you commit to something and you’re so down the pipe, you realize that, oh, you went a totally different direction. It’s great. That was amazing that I was also thinking in the similar vein on what you were trying to say. I wanted to jump back to some of the things that you were talking about, engineering tools. I wanted to understand what is it that you use the most or have used in the last decade that you feel is a phenomenal technology that changed the game for everybody?

Jason Valentino:

It is GitLab, GitHub. It’s so simple. It’s so boring. And I’ll tell you this, I know that’s a boring cop out answer, especially for someone that spends very little time in either. If anyone follows DevX and has looked at the space framework and surveys and stuff. Previously, I’m a big fan of DevX surveys. I think you get some of the best data you’re going to from just asking people, how does this work? How’s your day? We used to cheat at Peloton and we used to also add a section that’d be like, “Hey, check off which of these tools you use.” And it would be like everything related to software development in the entire company. And at the top of that list, over and over and over again, it was always GitHub. It is just like this is the one thing that you… A, we don’t have a lot of custom work on the implementation, obviously, but it just out of box does what everyone expects it to. It’s well understood. It’s a necessity for everybody that everybody likes versus a necessity for everybody that everybody dislikes, hates or puts up with.

David Joy:

Describe for folks who are listening, what is developer experience? Maybe folks don’t get that. Can you simplify that for some people as to what is developer experience? Why is it important for enterprises to have something like that in practice?

Jason Valentino:

If you think about your engineers at a company like the size of the one that I’m working for or previously worked for, it’s probably the majority of your product. It’s the majority… They’re building your entire company around you. If you’re [in [inaudible] it’s a 100% true. Why not take the time to measure and to move forward an agenda that helps make them as productive, happy, and engaged as humanly possible. And so DevX sets out to do just that. In some organizations, DevX may be the group that solely does the product research around this and solely builds the strategy around this. In other groups that may be also encompassing of some of the tooling that you’d want that team to have direct control of. In my world, it’s a little of both. We’re working now to get a little bit more of a product feel on it, but make sure interviewing developers, we’re getting… We’re running surveys, we’re looking at actual data from what’s happening in Jira, what’s happening in GitLab, what’s happening in our CI tools to see what the State of the Union is.

Taking that, distilling it down. And then for the things that we have direct control of, so for me, that happens to be CI, the SCM. Okay, I know how to solve these problems. I have direct control. It’s very easy to change things you control. The harder part of it is, you’re not running the rest of the company. How do you influence others? How do you take that data in a really compelling way, put it in front of the rest of the organization so that they go, “Ah, you are right.”? The way we do X is problematic for the goal here, which is developer engagement and developer enablement.

David Joy:

It’s been interesting too because last 10, 15 years, there’s been a shift in trends. We saw the cloud platforms coming and then cloud platforms will have their own specific products and databases and services. And then we had a phase where we had open source technology boom up. We had projects like Spark. We have projects like, Cockroach is an open source project as well. Lots of open source projects and developers want to try these out and use these in projects. But 10, 15 years ago that open source was not something that would relatively be easy to squeeze into any company, be alone, a financial company, they were so critical. Did you have phases where developers came back to you and said, “No, we need to use this open source project? This is really good. This is the direction.”? And how did you go about considering that into the overall developer experience?

Jason Valentino:

For tool selection and what should we use and what can we use. Normally on that interview question, I just immediately punt to the paper, the Netflix golden path, which is the story of, Hey, maybe I’m a regulated… In my example, I’m a regulated industry and there’s a little less regulated probably, but there’s a lot of tools. Tools are good. I don’t want to be the person to say, “No, you shouldn’t use this for your use case.” I don’t want to be that one magical architect that needs to review everything that goes in a company. What I’d rather do instead is say, “Hi, my job is do Jason’s stuff. This seems like a tool that falls into Jason’s stuff’s arena. I have all the data on what people like using, what people don’t like using which tools cost more, which tools cost less, which we find that developers are generally effective with.”

I’m going to build this paved path for people that want to use this tool that meets 80% of the need or whatnot, but I’m not going to stop people that are in this strange other 20% from doing their job. Just know the barrier of entry is going to be harder, because I’ve already covered all this thing for the people that adopt this. And so I like that the golden path concept from Netflix in that it’s standardization to make things easy for people that choose. The standard doesn’t squash the great ideas and the innovation that comes from folks that need to go a different path.

And then in the event people go down that road and find that weird database or find that weird tool that we have something similar in the company, but that doesn’t meet a need and it shows value through their work, then that becomes the next thing that also falls into the standard list. It just doesn’t start that way.

David Joy:

We’re talking about cracking a bunch of eggs in the process, which is expected to have when you think about innovation as being central to building out a cool product. And what I was trying to say with that is you’ve had phases, and then we had Kubernetes, and now we are at phase where we have generative AI. And I know you are working at BNY Mellon, so anything generative AI and codes getting used can be tricky, especially in the financial space. But lots of developers today are trying to use, say, Microsoft GitHub Copilot, and there is a lot of value that the GitLab in really good boilerplate code. Has that come up or are you thinking about how that fits into the overall developer X story?

Jason Valentino:

Oh my gosh. Yeah.

David Joy:

Can you expand that a little bit?

Jason Valentino:

Not a little bit. I think Copilot is cool, FYI, but anything that’s going to make the people’s jobs easier, anything that’s going to be helping them build portions of their code and doing it in a really innovative way is going to be good. And I won’t go ahead and I won’t tell you what pilots we’ve got underway at the company, because I don’t think I signed the right communication form to do so, but I do see generative AI, even the evolution of prompt engineering.

David Joy:

Prompt engineering.

Jason Valentino:

Super cool. I think it’s all win. It’s like getting to a point where all the care and feeding that needs to go into, we’re going to say healthy quality driven code. Some of it starting to magically appear around you as you’re working. It’s fantastic.

David Joy:

I was just talking to… In the morning today, interestingly, I was speaking to another person who is also from Austin, Texas. I’m not sure if he’s your neighbor, but we were talking-

Jason Valentino:

You don’t go outside right now. It’s 108 degrees out there.

David Joy:

Oh, yeah. It’s pretty damn hot here in Dallas too. But when we were talking, he was also mentioning this whole push for using generative AI, but in that whole experience there, he mentioned something that was really profound. The fact that generative AI can give you a really good function if you ask for, “Hey, give me this function.” It can give you that really good function and it can accelerate your code timing from say, 30 minutes to say 15 minutes, but you still have to spend another five minutes just to refine that and turn it into what it needs to be for your enterprise. Do you think that enterprises today who are talking about generative AI are also considering putting proper guidelines to allow innovation yet at the same time make sure that there are checks placed and to make sure that we are not just copying paste code that’s been written by somebody else at some other company?

Jason Valentino:

I would assume so. The cool thing is I think most people are starting with things like Copilot, which are going to come with some rails right outside or coming on rails. It’s not like just be like, “Hey, ChatGPT, could you write this for me?” I think folks are going to… I think the bigger companies are very wary of sending any question out to a GPT engine out in the middle of nowhere and seeing what comes back. But if folks are starting smaller, I think they’re safer. I’m sure as this evolves, the companies that love controlling things, will find various control points or they’re going to have to put in it. I’m sure the startups will just be like, “Whatever it runs.” There’ll be a good mix in between the two.

David Joy:

I think so. I wanted to also understand, your work at Peloton, what you did, that was a very interesting role because you were there to bring this whole team that’s been running at such a fast pace and there is so much demand for the product and you need to innovate, and developers are just going crazy using different things. What was that role like in an environment where there was suddenly a fire and you were like, “Damn, this rocket ship is not slowing down.” How did you work on that?

Jason Valentino:

Oh, I don’t know. It was fun. The cool thing was is I inherited a lot of the folks that helped get it off the ground. It wasn’t like I showed up with a team of 20 new people that were like, “Hello, Peloton, what are you?” And it all happened in logical succession too. As the number of developers grew, the ability to get things done safely gets harder and harder and harder. There’s no more just like, “Well, hey, so-and-so built this part of the code base, let me call them. So-and-so built this part, let’s call them.” The app became more critical. The app became more complex. And right around the same time, the company was realizing maybe that every post IPO ex-startup, this big monolithic leftover code from the time you were running as fast as you possibly could. And they were also in the process of, “Well, let’s figure out what the standard microservice should look like. Let’s figure out what our new stack should be. We know a new stack will unlock velocity, and the old stack is something we’re going to have to figure out a strategy around.”

And so I was fortunate in that DevX was something at the other devs in the company were asking for. Building the DevX function, fairly seamless. People were waiting for us to show up, and that was fantastic. I sat in with every VP in the company in my first two weeks, and they were all in unison about what I should work on first. And between that, doing the little things that a company that gets to that critical moment needs getting our internal platform product team together, getting our tech learning team together to help with things like onboarding and teaching the new skills that were necessary to move to these new stacks. All of it was really… I felt super welcome there. Super welcome. They pulled us in at the very right time, I think. If they grew more, DevX might probably would’ve been harder to introduce. If it was any sooner it might’ve been overkill for five devs sitting around looking at each other.

David Joy:

It’s always good to come into an environment where everybody feels there is a need for something rather than somebody saying, “Hey, we need this and this guy’s going to build it.” You know what I mean? It’s just much more friendlier I feel, that way.

Jason Valentino:

Fortunately, DevX is a real for… Good devs expect this. And so it’s one of those where whether you find a way to do it local, find a way to do it, just push a little bit of culture, it’s something that no one’s going to be like, “Yeah, we don’t need that. You know what I hate? Is someone caring about how I feel and how productive I am. I despise that.”

David Joy:

It’s funny. There’s so much happening in the tech space in the industry. Where do you go and keep up to date with what’s happening? You have a very important role now with BNY and every other role that you’ve had, there’s some sort of significance and you also have different stuff that you’re doing in your life. Where do you go to look for inspiration, look for stuff that needs to be learned that you can apply within your enterprise?

Jason Valentino:

For me, it’s because I’m working at companies that aren’t Google. I read a lot of the white papers that come out of the Googles of the world, like the DORA research teams of the world, because they’re solving problem… They’re sprinting and solving problems since they’re sprinting. And I’m working on a company that at the moment needs to get out of the crawl phase. And so for me, I unfortunately don’t have to be, but so much of a futurist because people are solving this for way harder problems. And I’m looking and I’m reading those papers and going, “Oh, either that’d be a nice problem to have, or that is a genius way of approaching this. I’m totally stealing that and adopting it, and I’ll give whoever wrote this paper credit at the end of project.” I do a lot of reading, just random papers on productivity, productivity studies, Abi Noda’s engineering enablement podcast. Amazing.

I’m a big fan of Gene Kim’s Idealcast. And so I try to spend maybe just an hour or two a week just learning from people that are doing it better than me, most likely, and getting ideas. And with my new team now, it’s like we have a little blog book club too. It shouldn’t just be me. I’ve got 200 folks that are brilliant and have better ideas than I do. And just taking turns sharing. What was the last cool, interesting thing you ran into? What technology are you guys paying attention to? It goes a long way.

David Joy:

The fascinating thing about what you just said is also that you have so many people who are also actively trying to get better, learn new things. And it’s about the challenge also persists where how do you take all these 200 people, get the best five ideas, then translate that into that one idea that will work for the company. Do you get into things where you see two good ideas and you’re like, “Damn, I want to apply both of them, but I just don’t know how to you make that decision?” It’s a good problem to have, but go ahead.

Jason Valentino:

I generally just want to hear, with the team, especially the size of mine, it’s more… And my old boss at Peloton, Jim Haughwout was the one that taught me this technique. It’s like, I want to hear their stories from the future. It’s really easy to sit and try to prioritize two things. What are the two next steps we should take? You don’t have a grading model for what to do next, but if you just actually sit down and instead say, “Where do we want to be in three years? Does my job even exist in three years? Let’s start there. What does the average developer’s day look like? What does the average database unit, like DBAs day look like? What does the actual org look like? What are we doing? What’s going to be good? What does fast look like?”

And then you get an agreement on what the future’s supposed to look like, the ideal future, and then all of a sudden you find people just start aligning much more easily towards what to do next. It’s no longer hard. It’s like, “Oh, if we’re getting there, here’s the sequence.” I don’t know if that directly answered the question, but-

David Joy:

I get the feel of where you’re going for it. Basically what you’re saying is that the thing is for you to understand what are those key things, and then once you identify that, everybody aligns to what that one thing needs to be. Got it.

Jason Valentino:

And we do that through everything. It’s like our KRs. Not just the KRs themselves, just what’s the story? Hey, three years from now, a new BNY engineer sits down. How quickly do they get to their first PR? How seamless was setting up their IDE? How quick is it to get a technical inquiry from another team to find out how to integrate with their API? You start going through those and then you start looking at the metrics. What of those things is the hardest today? And they start to define themselves as the most important thing to probably chase.

David Joy:

Got it. Do you think that having cloud platforms and having these ready-made, already maintained SaaS products in the market, do you think that has added value to developers so that they don’t have to think about, “Okay, I just need to know how to use this product rather than learn how to deploy, manage, run.”? Do you think that’s helped your team or teams that you’ve worked with before?

Jason Valentino:

Some. It depends. If you’re talking about just cloud technology and maybe the layer on top of it, absolutely. Things that help CD go fast. Very good. I’m hesitant to say I’m in love with CICD out of a box yet. I haven’t found a product yet. And there’s ones that all have great components of something that I want. There’s not one where it’s like, if I just did this, there’d be no customization necessary. You click commit and then life is great. That doesn’t exist yet, but it will at some point.

David Joy:

Talk to me about, is it like Netlify, Amplify? Have you explored those products that do CICD automation and deployments for you? Have you explored any of those?

Jason Valentino:

Amplify. Amplify is cool? I think Peloton is a customer, and they use them for the front end deployments. I think all of them are good for a single use case, or maybe two. I’m really waiting for one of them to just sweep in and be like, we own your entire deployment suite. I just don’t see it just yet.

David Joy:

What are the other tools? I know there have been some interesting tools in the space right now that I’m exploring myself from my stuff for managing. Do you think Notion… Have you explored Notion by any chance as a productivity tool?

Jason Valentino:

I have not. Mm-mm.

David Joy:

Okay.

Jason Valentino:

What’s it do? Should I know about this? Am I calling myself out in public forum that not knowing what it is?

David Joy:

No. Well, I don’t know if… We have Confluence as one of those places where folks go put stories and they can put their documentation and things like that. But I feel like I have seen Notion as this new platform that’s AI supported that allows you to do whatever you want to, you can create… You have multiple templates available and folks can use it for their own personal tracking of things and project progress and things like that. I found it pretty interesting. I’ve been using it myself. Maybe something that I can share with you that you can go and explore. I think the website is called notion.so.

Jason Valentino:

I don’t have a workstation here now, so I’ll definitely look at this.

David Joy:

No, it’s awesome. Well, Jason, I wanted to ask you, as we come to the close of… You’ve been so generous with your time and sharing all these amazing ideas. What is your ideal advice to anybody who is in this process of trying to build platform engineering or great SRE teams as to what are the few things that they should be doing to build a great team?

Jason Valentino:

I think when it comes to just anything platforming in general, I think getting real product managers involved is probably one of the biggest game changers you can… I see so much… Companies all kind of want the same thing. They want to develop great stuff. They want to make sure they’re promoting reuse inside the company, be that open source or other. And I think what a lot of us get hung up on the question of, how does adoption happen? And I think it’s very easy when you have the hammer, you have the power to just order adoption of your platforms to say, “I built this. I built this. It’s going to save the company millions and millions of dollars. Adopt my thing. What is wrong with you? Why don’t you like this?” Be that a pipeline piece of software, this, that, or, and another. And I really just… I don’t see success in that unless you’re actually measuring what’s the NPS score of the thing I built for what I want to build?

What are my product managers telling me the biggest problems are inside my organization? Can I actually make people happy by existing? Once you start to unpack that a little bit, I think you start moving. You can’t move in a wrong direction at that point. I don’t know. It’s like, do you want to be the DMV? Do you want to be the platform that everyone is really proud of that talks to their peers in the industry about that says, “This is how we do it here and this is great.”? And the devs love it. The customers love it. Everyone’s happy with it solves this problem. Long story short, I don’t think there’s enough. And this is coming from someone that’s done nothing but chase bits around all his life and has no formal product manager training. I think figuring out the voice of your customer in my line of business, that’s the developer. Figuring out that voice and working backwards from it is probably the most important thing you could do.

David Joy:

Where else do you think… You have shared some amazing advice. Some really great comments I would say on this podcast. And for everyone listening, Jason’s amazing. He’s got a pretty, I would say, decently active LinkedIn profile. I’ve seen that you’ve interacted. Where else can people follow you? Is that a blog place or Substack or something that you do on LinkedIn where people can follow you and catch you?

Jason Valentino:

LinkedIn’s probably the most useful, and then occasionally I’ll remember to push the post to Twitter button on LinkedIn. When I’m publishing something primarily on LinkedIn. That’s generally going to be it. And I try to pin, like this podcast, I try to pin stuff that I actively authored. I just have gotten lazy at it. We’ll get better. This will inspire me to be better. Also, David, I don’t know about you, but when the whole pandemic happened, I didn’t make the transition from onstage in conferences to doing this. It isn’t until after COVID, I’m like, “I guess I could start talking to people again. This sounds like fun.”

David Joy:

Generally feel like everyone who has been in this space, especially last 10, 15 years, who’s worked on innovative products such as you, has so many great stories to share. And if I have to leave you with anything, I feel like I just would say there is more Jason needed in the world. We need more of you in the sense you have great experience, how you have been inspired by what Netflix has done. I think some of the things that you have done, putting it together in a place, like how you shared in today’s podcast. It just helps so many people. And that’s how great communities, great open source communities, and great ideas are passed around. I feel like we need a little bit more of that. And I’m glad that you took the time to come on the podcast and talk about it with us.

Jason Valentino:

That’s awesome.

David Joy:

It’s awesome.

Jason Valentino:

Thank you, David. You’re way too kind.

David Joy:

I am kind, but you are too good.