In this webinar, we talk with platform engineering leaders at BambooHR, a SaaS leader in the Human Resources space, about furthering their journey into Kubernetes. We are joined by Kelsey Hightower to help moderate and provide commentary on what he has seen in the space. As BambooHR kicks off their journey with Shipa, learn from the prospectus of the team moving the needle in engineering efficiency and developer experience.
Lots of great topics and pragmatic advice covered along with questions from the audience. Feel free to catch the recording in the blog post / here.
Topics:
- Build vs Buy
- Contracts between users
- Environments at BambooHR
- Dealing with lock-in
- Implementing developer abstractions
- Shipa and Kubernetes
Transcript
Bruno Andrade 0:08
As organizations move towards cloud native, we see there is an opportunity to talk about how do you treat application as a first class citizen? And how and what are some of the strategies to tackle that right and improve security, productivity and so on. Here with me today, I have the the team, some of the team members from from BambooHR, Kelsey Hightower, and some of us here from Shipa. I would love to have some of you folks introduce yourself, maybe we start with that with Dave, Dave do wanna give a little bit of a background about you?
Dave Petersen 0:43
Sure. I’m Dave Petersen, I’m the Director of IT Operations at BambooHR. I’ve been with Bamboo for just over a year. But I’ve always been a fan of the company. I’m good friends with the founders. And so I’ve, I’ve, I always say that I’m, I’m a longtime lurker of BambooHR. So I’ve, I’ve watched the growth over the last 10 years of the company and excited to finally have a spot available for me to join them in, in their journey. Previously to that, I worked at another HR type company called O.C. Tanner, where I did similar things. And we were instrumental in kind of bringing the software in development pipeline and platform into the future from a very old and waterfall type development methodology.
Bruno Andrade 1:41
Nice, nice, thanks. Thanks for the time, maybe, Kirk, you want to go next.
Kirk Hoganson 1:47
I’m Kirk Hoganson. I am the Systems Architect at BambooHR. I’ve been there less than a year. Before that I worked for a large financial institution, we were implementing, starting to implement Agile and containers and those methodologies at a large scale for that enterprise, and then came to BambooHR, which are last year to start. Implementing that journey, that journey for BambooHR to help them start to see realize the vision of containers that they have.
Bruno Andrade 2:20
Nice appreciate that. Before we move, we move on to Kelsey. Maybe no introductions needed there at this point. But myself, I’m one of the founding members here at Shipa. As you can see, a failed musician, I couldn’t make life as a musician happen, I was not good enough to make the cut, moved on to software engineering, not good enough to make the cut but too old and losing hair at this point. So the rock star life is no longer a possibility. So moving on to cloud native. And with us today as well. Kelsey, who sees so many users going through this journey. So Kelsey, do you wanna talk a little bit about yourself?
Kelsey Hightower 2:57
Kelsey Hightower been around the game for a long time, created some useful things, I hope in the open source community. And since the days of configuration management through the transition to DevOps to where we are today, where we’re kind of faced with this big challenge of buy versus build, right? We know, the ultimate goal of these platforms is to run the applications that people build. So are we there yet?
Bruno Andrade 3:20
Nice, nice. No, I appreciate you taking the time and sharing some of her thoughts today. with us.
Kelsey Hightower 3:31
I’m gonna I’m gonna play the role of moderator today. My goal really is to try to ask questions, I always like hearing from customers, Bruno could probably attest to this as people who build things, we really believe that the thing we’re working on is the solution to most of the problems in the space. But I think it’s important for the community in general, to really hear for from people who actually had to make a decision that has to actually run in production. So I’m going to kind of lead some of the q&a. We’re also going to be doing live q&a from the audience. I think what we’re going to do is, if you have a question, I’m going to pay attention to the q&a section. And I’ll interweave your questions in throughout. Okay, so it’s how we’re going to try to do it today. So I guess maybe at the top, maybe some people don’t know what ship it is, maybe quickly, Bruno, could you just kind of describe Shipa, but then I’m going to kick it over to David and Kirk around why did they choose it for their particular situation?
Bruno Andrade 4:22
Yeah, we’re so here at Shipa. I had an opportunity to come from the cloud native space as well previous to that. And I realized that there’s a big opportunity to bring an application level layer or API to folks that are deploying applications using micro services and Kubernetes as as a platform, because Kubernetes is awesome. But as it exposes a lot of the components to developers and platform engineers, it creates a lot of complexity as you scale. So we saw an opportunity at Shipa to create that application platform that helps bring that application layer visibility that enables kind of developer productivity and For platform teams to focus on on security. So that’s, that’s our focus today.
Kelsey Hightower 5:05
Yeah, remember, when Kubernetes came out, I had a saying, or I think I did a tweet. And I said, you know, Kubernetes is a platform for building platforms, it’s a better place to start, not the end game. And so now I’m going to kick it over to you kind of Dave or Kirk, either one of you can take it, maybe we start with Dave, you know, I’m pretty sure you’re looking at all the hype, maybe there were platforms you were using before Kubernetes, maybe you had a transition from Kubernetes, to something like Shipa, what made that decision versus like building your own gluing together a bunch of components, or downloading and installing something like Shipa in order to deliver that app experience to your developers.
Dave Petersen 5:41
You know, when when I started looking at stuff like this several years ago, there wasn’t really a lot out there. And so the builder vs buy equation was very different than it is now. You know, but one of the guiding principles is something that it well, it’s called the Purpose Alignment Model, you can look it up on the internet, it was developed by one of the persons I worked with for several years, Niel Nickolaisen. But the principle behind it is, is that from a software engineering perspective, from a company perspective, you want to spend as much of your energy and resources and innovation on building stuff that’s going to be innovative for your company that’s going to make you market differentiating in whatever space you’re in. And so you don’t want to spend a lot of time just doing things that give you parity. And so so when you look at stuff, of build versus buy, you really want to look at it in terms of is this, if I build this, will this give me a difference or a competitive advantage in the market? If the answer there is no, then you want to look at, well, what can I buy, then that will help push me forward and help improve the velocity at which I can deliver innovation for my company? And so that’s kind of the way I look at all software, when it comes to delivering applications. Is there something out there in the market that will give us a way to move forward from a standards perspective, from a best practices perspective, that will increase the velocity or speed, the value of that software engineering team that’s actually creating that, that market differentiation for our company.
Kelsey Hightower 7:36
Before I swing it over to Kirk, you know, what do you say to the people like, Look, man, we can’t just use a standard solution, we got Oracle, MuleSoft, I got mainframe still. So anything I do from this point going on, it’s gonna have to be custom, because we’ve tried off the shelf solutions before and it just doesn’t do everything we want. What do you say to people like that?
Dave Petersen 8:00
So So yeah, it is tricky. Because sometimes there there’s this huge technical debt or legacy type application that you still need to support. And so you do have to do some type of glue development in the middle that kind of brings that old stuff along to whatever the the newness is. But really, you should look at what are our mission critical activities? And how do we bring that up to a place where it’s, we’re doing best practices, where we’ve embraced standards. And we’re doing you know, what is parity in the market. Because if you get to that point, that starts to decouple you from some of the legacy stuff or the things that you depend on. Because now you have an industry standard, that you’re building towards that, then you can use any industry standard tool that’s out there that better fits your needs. But it is a journey. Sometimes it’s gonna take years to get that point to that point. But your focus should be getting to the place where you have decoupled yourself from having to do custom code just to run your applications.
Kelsey Hightower 9:10
Kirk, I’m going swing it over to you, I mean, similar question. You know, when you think about number one, has it worked? Right? This thing has been brought in my guess is that may be some friction along the way, I’m pretty sure there’s a roadmap of things you want to accomplish, maybe give people a sense of now that you’ve made that decision? How’s it going?
Kirk Hoganson 9:31
We’re still relatively early in the journey here at BambooHR, moving to containers. And I think one of the things you have to consider is what are not only what are the business goals and alignment, but what are our particular resources. So BambooHR is essentially a software company. And so we have some great engineers. We have excellent engineers, and that’s a great resource for us to use. At other places you may think it’s better to build to buy something off the shelf. But if you have bespoke needs, and a lot of people do like went out, it’s, it’s important to be considering how those line up with what you’re actually trying to do and what your resources are. So it BambooHR. My opinion is that Kubernetes is sort of killed the the PaaS and then now became any ecosystems like the PaaS is dead, long live the PaaS; Kubernetes is risen sort of fill that gap with more of an ecosystem, which takes more work to configure, but it has a lot more flexibility than sort of a one size fits all solution that you might have implemented 5,6,7 years ago, if you were starting this journey, now you have a lot more options. And that enables a lot more flexibility. But there’s a lot more complexity that comes with that. So when we started looking at this, we started breaking down what do we what are our resources? What can we put in? What do we need to achieve. And with that in mind, we started looking at well, one of the biggest problems we have is we started this journey is the communication between systems and development and systems engineers and software engineers, we have good communication, good relationships, but every time you have to go back and forth between the two parties, what you’re doing is you’re adding time to the process. But for a software company like us particularly, we’re in the HR space, which has had a lot of changes over the last couple of years with the pandemic, COVID, there’s a lot of HR related changes. So we need a software company for our product, to be able to stay on top of those changes. So we need fast development cycles, we need to be able to provide new features quickly. And to do that, you have to get as much of those touch points back and forth out of the way. Because every time so Software Engineering has some information system engineer has other information about the the infrastructure. And you have to every time you go back and forth, if there’s some information that needs to be shared, you’re adding time. So if you want to develop a new service, it takes a lot of time to go back and forth that oh, I need this piece of information I forgot or, and to communicate about the infrastructure, and then the line between what each party is trying to do it gets a little blurred. It becomes difficult. What you end up with is projects that actually have if you could automate it have minutes worth of work, and becomes weeks worth of work because of the the personal back and forth and the information sharing and the scope gathering and all that. So we started looking at how do we speed that? And automation is a top of the list and automation. We started looking at how do you automate the deployment of new services on Kubernetes? Or maintenance incriminated updates, there’s a lot of moving parts. If you look at Kubernetes application definitions, these YAML’s like, you know, 500,000 lines long, how do you automate the generation of that? Well, it’s just, it becomes almost impossible to do that with any sense of flexibility that doesn’t lock you in. And that’s where something that provides some kind of abstraction layer. So that’s how we ended up looking at Shipa, we’re looking at a few different things. There’s some other stuff out there. But it’s a relatively new demand in the market space, there’s a lot of products, offering that in a way that isn’t trying to do everything for everybody. They’re [Shipa] just covering this abstraction layer, so to speak for applications. If you can build an abstraction layer, suddenly, that 1000 lines of YAML or 500, lines of YAML, it can shrink down to 20, 50 lines, suddenly, I have something I can I can automate, I have something I can build some systems around. And I can create those YAML definitions, that reference that have framework references, or these interaction references, that reference much more infrastructure in the back end, but that apply to my resource limitations, or my network requirements or auto scaling rules. And that makes everything easier to automate, which helps speed up the process because it gets the human interaction between the different parties. It minimizes that and it allows things to go a lot smoother, a lot faster, which helps the whole development cycle.
Kelsey Hightower 14:06
Yeah, so I think for people that may not have a lot of Kubernetes experience, when you’re hearing kind of this need for automation, and I know you mentioned abstraction, think about like your database, right? Think about your raw database, right? You put data in there, and there’s a bunch of tables and schemas, there’s foreign keys, there’s all kinds of things that you put in your database. But we never think to give customers direct access to our database and let them just run random queries, right? That’s just way too much information. And also, they may just get in trouble by doing things that they probably shouldn’t do, because you’re exposing the entire database surface. So we tend to create API’s, right? So think about those apps that you write that sit above the raw database, so everything is still there, all the capabilities are still there. But you know, for the target customer, you only need to expose the API create user, delete user, not select star from database where users equals that’s that’s too much information, and then becomes way too much work to communicate without these contracts. And I think maybe that’s what Shipa brings to the table is this contract that’s a little bit more defined than the big wide world of Kubernetes. And I know when I met Bruno, for the first time, I’ve been advising companies in the space my days at CoreOS. And I always knew that Kubernetes was going to need some better abstractions, it was going to actually be useful to more people. But I want to get your thoughts on this. I know, it’s like the struggle between hiding the complexities of Kubernetes without losing the power of Kubernetes. Like we’ve seen in previous attempts at PaaS, what was your approach to thinking about building Shipa? With those two things in mind?
Bruno Andrade 15:47
Yeah, that’s a great point. So when we started looking at Shipa, we, we thought about taking it a different approach by try to detach the application from the underlying infrastructure, right? Because we I completely agree with both Dave and Kirk. And one of the challenges I see is especially mid to larger organizations, you’re going to end up running today, you have one infrastructure as code solution. Tomorrow, you might add another one, because on other teams require something and then another team is going to require another CI pipeline, for example, another team is going to try to play with another provider and cluster inversion. So complexity is never going to go away. So how do you actually detach your application from that complexity is the opportunity we see to kind of speed up to development, speed up the interfaces that expose the writing interfaces to the developers in a way that they can consume, what matters to them, and application definition. And, and that’s part of what Kirk mentioned, for example, you’re going from those 1000s of YAML, lines to to dozens of Yamo lines or to more if you’re using HCL, or you’re using JavaScript, because you’re using other tools, then the same definition keeps consistent across the same the different tools. So now you’re detaching things in a way that you’re evolving your infrastructure and testing and experimenting with the different components. But you’re not affecting that developer experience, how your applications are deployed, and managed and secured. And honestly, I think that’s, that’s that’s the direction eventually Kubernetes will become that implementation detail and use a platform engineer, you should be able to choose the components you want to use by your your application, like Dave mentioned. We talked to one user that is interesting there. The way he position is like we sell food, that’s what matters to us, right? If if the sell food application is running on a mainframe, the developer shouldn’t care my end user who was buying food, they don’t care. So how do we actually detach those two things and provide that API that allows you to keep selling food faster, right? So that’s kind of where the principle where we come from?
Kelsey Hightower 17:47
I want to remind people in attendance today, if you have a question, you can drop it in the chat. Ideally, you use the q&a, so I can actually tell what things are actually questions. And we’ll weave them into discussion as we go. I want to kick it over to this other thing, like, number one, maybe to Dave or Kirk. How do you think about the lock in situation? Right? Like, I think one thing we all worry about, even when projects are open source, or proprietary or SaaS based, we always worried about over correcting towards this off the shelf kind of approach, even if it’s an industry standard? How did the team process this idea, that man, what do we bet on the wrong horse, and we end up losing the race, maybe we start with you, Dave.
Dave Petersen 18:32
I used to worry about that a lot more than I do now. Reality is, is that you can get a lot of mileage out of just picking a direction and then optimizing that. And if it ever gets to the point that you do need to switch then you just switch. So I don’t worry too much about locking. But maybe one of the reasons why is also because I do think as long as you’re thinking about a clean architecture, and the overall arching goes goals of making something that’s portable, that’s app centric, you know, in that, like Bruno was talking about it was a one kind of abstract a lot of that away, so that the interface that our our end user, the software developer has really doesn’t change and we could change almost everything underneath. And it doesn’t matter to them. And so I I think if you if you keep the goal in mind of being nimble or agile in the fact that you are using standards, you’re communicating in standard ways, then you can swap out components without much of an impact to the end user. And, you know, in my experience, walking really has not been a problem. I’ve always been able to move stuff if needed. because we always tried to be, you know, like I said, as standard based as possible, and we abstract where we need to so that that that end user interface does not change.
Kelsey Hightower 20:14
Kirk, you know, I, I’m pretty sure people watching this, like man what environment? Are they running in? Is this all on prem? Is this in the cloud? What even gave you the ability to make this kind of decision and actually start to roll it out, even though you’re early in the journey, so maybe give people a little bit of insight, as you kind of tackle this locking question. Also, maybe give people more insight about the environment and the type of people that you’re delivering the solution to.
Kirk Hoganson 20:40
So here at BambooHR we are entirely in the cloud, we’re sort of a newer company, software company. So So I worked at large scale institutions that have a lot of on prem. And that’s a totally different kind of challenge. Here, it’s been, which are we’re totally in the cloud. And that was something that that was done a long time ago. And when we’re trying to balance, something like lock-in and features. So there are certain features, if you’re in the cloud, you’re in that provider, they have certain features that you may want to leverage, you’re there, it’s easy, you’re paying for it already. Some of them, how do you find that balance. And it’s important to me to think about it as a balance issue, because there’s a different type of lock-in when you start getting custom solutions, we start writing your own stuff, you can actually end up having a different kind of lock -in is different than a vendor lock-in that you might have, from some kind of a off the shelf solution. So like Dave said, it’s about, we’re trying to adhere to some kind of standards, but at some point, you just have to pull the trigger, so to speak, you say, well, we’re, we’re leaning into Kubernetes, we’re deciding on this. And when you start making those decisions, say this seemed like a good bet for us based on our business. And the environment we’re in, we’re in the cloud, instead of on prem, this works Kubernetes is a good place to start for us. Okay, now, that introduces a series of, of solutions that are available, and also challenges. So you start working through the problems step by step with the business goal in mind. And then you can find a balance between your sort of custom lock-in stuff where you’re writing your own solutions that end up kind of being less portable than you might like, and the vendor that your your vendors that you’re using, whether that’s you know, AWS or GCP, or Shipa. And you say, how do we how do we manage to have both of these in the solution sort of starts to present itself, because as you make decisions, like I said, you end up with solutions presented and problems within these you is your tackle each problem with the solutions that you have available, you end up with something that is a little locked in, it’s always going to be a little locked in, right. But if you have some standards, like Kubernetes is more portable, so you’re saying, Well, we’re on Kubernetes. And maybe we’re using something this this using Vault for secrets, for example, and we’re using Shipa to handle this kind of abstraction. Those tools are portable, and then you can, you can’t exactly lift and shift. But you can think about how you can take these core components. And if they’re modular enough, put them move them if you need to, or duplicate them elsewhere, if you want to be multi cloud, but things happening now. So you and then the pieces, you’re able to keep the the custom pieces a little bit smaller and less locked in. So maybe you have to write a different custom layer of automation, if you’re on AWS and GCP. But if the back end pieces like Kubernetes, and Shipa, or Vault or whatever are the same, then those changes are small, and you were much you’re much more portable at that point. And it’s it’s really thinking about which standards apply, like Dave pointed out.
Dave Petersen 23:54
I’d add you know, for the the huge win early on for us at my previous place of employment was realizing that the container gave us that portability. You know, if we could run something in a container, it was easy for us to deploy that on prem or in you know, any number of public cloud providers. Because it was, you know, that it was designed from the beginning to be something that was very portable and self contained. And so, you know, making some architectural decisions early on about portability and understanding where you could have lock-in and where it’s okay, and where it isn’t okay, and then choosing those technologies that will give you the most flexibility and options. As things change over time is is I think very key.
Kelsey Hightower 24:46
Dave and Kirk, I’d love to get your thoughts on this. You know, when people for years have been asking me about like the best way to get started with the ability to consume these platforms that we’re talking about today. I’ve always asked him like take a step back and at least get to the point where you number one can produce artifacts for your app. So if you’re writing JAVA get to the point where you can actually reliably produce a WAR file, if you’re writing on Red Hat look reliably make an RPM for your app. But then to open the door to all this other stuff, that container image artifact, if you can get to the point where you have enough discipline to reliably package and produce an artifact that can be consumed in a Cloud Run, a Shipa, or raw Kubernetes, I think that tends to me be like a precursor, I think a lot of people try to skip that step and say, Oh, we’re just gonna go do Kubernetes. And they actually have no path to getting their apps into the right packaging format. How important is that step actually? Or is that something you can just think about later, once you’ve chosen a container platform?
Dave Petersen 25:51
That’s a really good point. There’s several aspects to this. The technical aspects is just part of it, right? So saying, Oh, I’m going to do containers and Kubernetes. And then all of your problems go away, and magic happens. And you’re there will end in failure, because you have to have a culture shift, you have to shift your your development processes, and you have to shift your policies around how you develop and deploy code. All of those things have to work together. To get to this point where you like you said, you can reliably create an artifact that you can deploy in a container. And you also have to understand what that means. From a software architecture perspective, that is not something that that is really designed to have persistence. And so you have to understand, Okay, well, how do I deploy and write my application to take advantage of the portability of containers that something like Kubernetes, so that I can I, you know, there’s a lot of benefit to that. But you can also end up deploying something that fails horribly, because you’re not using the system correctly or as intended. And so it’s not just the technology, but it’s the processes and the policy, and even the culture that have to come along. First, in order to make you successful in using those technologies correctly.
Kelsey Hightower 27:20
We got some questions coming in from the audience, and they all seem to center around the actual problems you were trying to solve. So we’re gonna dig into that in a second here. I think one thing I want to do is make sure I think people just kind of know, the biggest thing, I’ve seen people fail with any of these initiatives, they can never get the base homework done. I’ve never seen them successively find the discipline, to make it a priority to get their house in order so that they can actually use this next level of automation instead of abstraction. So please, do not ignore this stuff. I’ve seen people get started waste a whole year, realize that they are not interested in going back and putting the right things in place that that can get those benefits for the rest of the stack. Before we switch to some of the audience questions. Bruno, I wanted to I remember when I first met the team, and I saw Shipa the surface of like, Oh, my God, this looks like Cloud Foundry all over again. And we’re going to create a mess that’s going to be hard to untangle. And the biggest questions I had that time were, how much of Kubernetes is still there, or the hood, if I lift up the covers, while I still see things like pod disruption, budgets, deployment objects, because if I decided to uninstall Shipa tomorrow, I want to make sure that I still can keep some of the core artifacts. What is the platform stand now? And what was your answer back then?
Bruno Andrade 28:43
So there are different points of view to that, right. The way, the way it is set up today is we focus on bringing that kind of exposing the right API’s or the right interfaces. So the right users, if you’re a DevOps Enginee or Platform Engineer, how do you secure your applications, right, regardless of the infrastructure, school, CI pipeline, or cluster deployment. And if you’re a developer, because one of the problems we we found a lot talking to users is developers, they don’t really know how to support their applications on Kubernetes. They go well, up to the point to the point that they generate the artifact. But once it’s deployed, that it creates a myriad of objects, they don’t really understand what a deployment set is what a service ingress rule and those becomes kind of components that it’s hard for them to grasp, because those are infrastructure level components, right. And that’s fair. So we were very concerned about exposing the right interfaces and for the developer, the right views that they can finally support their own applications. But to your point is, everything that we’re leveraging is Kubernetes native, even we allow you to as a Platform Engineer to choose is to or choose Traeik or Nginx or whatever other components are using infrastructure, as code and so on, and who created those interfaces for them. But if you lift sheep up You are going to be able to see all your objects in there, you’re back to the world of exposing infrastructure components. But you’re gonna see all your objects, all the configuration. So if you do need to break the glass, or if you need to, if like Kirk said, if you want to write custom extensions to address very specific use cases, you can do that. We always had the approach of not taking over your cluster, we just don’t think that’s the right approach, you should be able to own and deploy applications because we understand that she was not going to fit every single application, it’s impossible. So you may be running some applications in a namespace and a cluster, or you may have some custom extensions. And you also have Shipa there with the capability of seeing everything that is happening under the hood as well.
Kelsey Hightower 30:43
Yeah, so I think for people kind of listening today, when I first gave kick the tires on Shipa, I already had a running Kubernetes cluster, had some apps already deployed, I can add Shipa as another entry point into that cluster, gave it a few namespaces. And it turns out, things can live side by side. And also there’s a huge ecosystem. I think they brought this up earlier, I think maybe Kirk even doubled down on it. That ecosystem already has a bunch of tools that understands native Kubernetes objects. So one thing that you can actually benefit from is the fact that those things still work, I’m going to pivot to some of the audience questions, because I think this is the meat of why people came. The question goes, Can we we should get into the actual problems, Bamboo HR did solve; they want to get into some of the details around the actual pain points you were having before you installed Shipa or chose that as a solution. So we’ll start with you, Kirk.
Kirk Hoganson 31:45
So at BambooHR where we started with containers in sort of a prototype way, like we had some understanding of the fundamentals, but there’s a certain amount of this that you can’t really understand until you start to do it. We’re engineers, and we think of things in very technical and sometimes very linear ways. But something like this is actually create a process, you’re going to have failures, and you’re going to have to pivot, there’s going to be assumptions that you made that aren’t going to work out. And in every time I’ve been involved in something like this, it’s always been a part of that process, whether it was at a large on prem organization trying to move to containers, we had issues with the first platform we tried and things didn’t work, we had back a lot of stuff out and try again. But you’re learning lessons as you do that. And one of the things that happened when we started with containers is we ended up with one of my engineers has dubbed YAML sprawl, all these so many different Helm Charts and YAML files, and it’s and they’re growing, and who’s maintaining them and who has access to them. Because you don’t have any without any abstraction. Everything is so direct, that for every application, we’re generating a bunch of Helm charts from scratch. And that took a lot of time and add a lot of complexity. And a lot of those Helm Charts, for example, have a lot of duplication. So there’s a different Helm Chart for this app, and this app, but they’re both Kubernetes applications that could potentially live on the same cluster, and have a lot of the same underlying shared infrastructure resources. But we’re specking them out each individually. Because each one is sort of a bespoke approach. So what Shipa we’re using Shipa for at this point, as we’re as we’re roughly new with Shipa, but what we’ve started to do with it is to build those abstractions and frameworks and plans that will enable us to cut down on the maintenance and creation of all these Helm Charts and YAML files to reduce that workload, and to make it easier, and it reduces the amount of toil that engineers have to deal with. But it also allows us to then automate on top of that. So it’s about really the complexity when you start getting into it of all the Helm Charts and all of the ammo files and all the resource definitions, it takes a lot of that and simplifies it for developers or automation or even engineers.
Kelsey Hightower 34:09
So I want to ask a follow up question to that. Because I think some people would say, the simple act of adopting Kubernetes is what cause your problems, right? Like bringing in Kubernetes, meant that you introduced this huge surface of configuration. And then I know some people will go and reach for a Helm Chart that you realize that maybe even make this more complex with bringing in a whole config language that is Turing-complete. And now you’ve created such a mess that now you have to go solve it by bringing in something like Shipa so I think to this immediate question, Shipa reduces or solves the complexity of Kubernetes. That may be a follow up question would be why containers in the first place, right? Why not just stick the Puppet, Chef or Ansible or god forbid SSH bash scripts? Why Kubernetes in the first place.
Kirk Hoganson 34:57
That’s that’s really a great insightful question. Critically for BambooHR, we’ve grown largely on a more traditional model for a long time. And then by the traditional model, I mean, sort of the, the pets instead of cattle, the servers where we know all their names, and everything is sort of managed carefully, you have X number of servers, and you know, all of them, and you have to patch them all. And you have Puppet running to do as much automation as that can. But you end up reaching sort of some limits you sort of that that model plateaus, and it starts to be harder and harder to grow. And it starts to become more resource intensive for engineers, it gets harder to manage and automate that. So we’ve reached a point with our growing business that it’s harder and harder to scale. And also, we end up having to sort of provision for our peak load times. So we have, we have different workloads, the different peaks are based on what day of the week it is, and what day, the month, it is based on HR cycles. And so we have to provision for the peak because we can’t scale up or down fast enough with that traditional model. And our two is not fast enough to have to be responsive during the day to changing demands from our customer base. So the flexibility and scalability of containers is unparalleled with the traditional model. So if you want to grow, and you want to be flexible, particularly if you’re in the cloud, and you’re paying for every single compute minute, it matters that you’re able to scale up or down. And it matters that you’re able to do these things. Because otherwise you get kind of locked in and what you end up with, you’re slowly increasing your tech debt and your engineer toil, and that can get out of hand. And that becomes so unmanageable that then you can’t innovate anymore, because you’re just stuck dealing with the traditional system to have to constantly managing these, these pets, you know, these systems that are more traditionally managed, and it becomes harder and harder to do, what you need to do and innovate and scale and move quickly, particularly if you’re something like we are, which is a software company where development cycles need to be quicker, and they depend on the infrastructure in order to release code and develop code.
Kelsey Hightower 37:17
Yeah, I want to remind the audience, I’m going to switch to a different question for you, Dave, out of the sake of time, and you can definitely add a little bit to this one as well. When you look at Kubernetes. I know a lot of us see net new exciting in know all the buzzwords and hype. But the truth is, under the covers for most people. Kubernetes is just managing a collection of virtual machines doing very similar tricks that you would do with your scripts and automation tools. It’s just this universal check point where we’ve all that done that before have just given that pattern or name and the control plane wish to implement it. And then for you, Dave, you can definitely add on to that previous question. But this question here, I think is from one of the audience members. Can you share a few friction points you face in the customer acceptance and how you overcame those? So you roll out Shipa? I’m pretty sure there was some things that people found to be a point of friction. How’d you deal with them?
Dave Petersen 38:12
Yeah, so I just add to Kirk statement. As you as you grow, lease with Bamboo. And I think this is true for any company that grows significantly. Our software engineering teams practice, you know, Agile development methodologies. Agile works best when everything that a team needs can be done by a team member. And the more and more you grow, the more difficult it becomes to keep all of those dependencies on a single team. And that’s, you know, one of the other big pain points of the more traditional development or infrastructure model is that we can’t support as many developers because the more developers the more teams you have, the more divergent their needs and things that they’re working on becomes. And so we have to be as fast as flexible and as dynamic as possible. So that we do not cause any type of blocking because they’re dependent on us to provide a an infrastructure for them. And you can’t get that with a traditional infrastructure, you need something like Kubernetes and containers that can give you that flexibility and dynamicism that you need with a multitude of agile development teams.
Kelsey Hightower 39:36
And on the second question around like the friction once you introduced this, what was what was that friction and then how did you overcome it?
Dave Petersen 39:43
So fear of changes is always big. People get used to doing what they’re doing and they are it’s difficult to change because it’s different. And, and so a lot of it is just, you know, managing fear; fear, uncertainty, and doubt, there’s going to be all sorts of reasons why this isn’t going to work, why we doesn’t apply to us, you know, well, we do this thing, and it’s so special that could not ever possibly work on this new infrastructure. And so a lot of times, it’s just spent understanding, you know, what, what are real concerns versus just, I don’t like change, because that’s kind of the default mode for humans in general. And just dealing with, you know, actual problems, and, and resolving those. And then, also, when you think about what are you’re providing to the, to your customer or to your development teams, they’re always going to take the least path of resistance. And so where we’ve had success is, instead of focusing on your entire organization, focus on one team that you think you can work well with, solve their problems, and get them on that new solution, you know, with with an MVP, MVP type product. And then once you kind of have you know, that use case that works, you can see that one team’s adopted it, then people become more curious. And if it’s, if they can see how easy it is and how it solves their problems, then you start having people come and ask, Well, hey, can you help me with this? This is something that we’re struggling with, how can we resolve that and make it easier for us. And so just really, look at individual team problems, solve those for those teams, get them on your side, and then just kind of grow because it’s easier you you’ve made something easier than what they have now.
Bruno Andrade 41:44
If I can just add to that as well, in terms of the friction, one of the things we see with folks is the the perception of value, right? Sometimes you might think that the perception or value of your team or you as an engineer, DevOps or Platform Engineer, for example, is building those complex Helm Charts and all these complex automation, sometimes you you internalize that as the value that you have in your organization, when on the other side actually devalued, like Dave, and Kirk mentioned is, how do I enable my developers, which is basically my end customers to move faster? Right, so I’m moving from kind of building a bunch of toil or repetitive work and something that I can actually pushed to my end user to be self service, and so on. And now I’m focusing on scaling, choosing the right components, the right architecture, and the right security that should be implemented. So I think on the on the friction point, as well as how do you enable the right value, right and change the perception of value that you might have from a DevOps are from a development team, for example, that you were focusing on the right things that add value to you, as a professional into your company, if your developers move faster, I’m sure the DevOps team is going to be recognized at the end of the day, because you are enabling that, that movement.
Dave Petersen 42:57
Yeah, in fact, I, I argue quite a bit that developers have provided zero value to the company until they deployed their application. And so really, our goal should be, we need to do everything possible to increase that speed to deployment. So from the inception, or conception in that developer, or development teams mind of something to where they can deploy that is that’s where the value is. And so we need to shorten that time as much as possible. And it really becomes, you know, enabling their our developers to move out of velocity that’s just right for them. And that’s where we have the value for the company. And it’s not, like you said, writing Helm Charts or having them write Helm charts, you know, it’s really just decreasing that time to deployment.
Kelsey Hightower 43:51
Alright, we got about five minutes left. So we’re gonna go and maybe more rapid fire type of questions and answers. So I’m going to let anyone on the panel jump in on this one, when you’re designing and building abstractions around the common parts of your domain applications, or their apps that didn’t fit the new abstraction very well. What were those apps like? And what did you do to solve this?
Kirk Hoganson 44:16
Okay. Well, I think for us at BambooHR, we have a pretty limited tech stack that we work with. And that’s helpful for onboarding some new technologies, because we don’t have a lot of different languages we use we don’t have. So it enables us to do something an MVP that’s closer to what all the applications can fit. So I think when I Previous to this, we were to containers. The issue was the ecosystem around the infrastructure rebuild, and some of that is personnel and people like days if you’re change organizational problems, and some of that was just onboarding because this stuff is just a tool containers are a tool Kubernetes is a tool ship is a tool But if you can’t easily use the tool get access to the tool, it’s not very useful. So you can solve some of those problems by have been making the onboarding process easier by thinking it through. And As Dave said, if you can start with something that does fit, not everything, if you have a large organization, a lot of different types of tools, not everything is going to fit right away. But you start by solving the problems you can solve, and then you grow from there. So you’re not going to get a one size fits all solution right away.
Kelsey Hightower 45:31
I think it’s also helpful to remind ourselves that most applications, kind of take data and do something in the middle and export data somewhere else. So make sure you’re understanding that I think the traps we’ve fallen into in the past is that we start doing weird stuff like coding to POSIX file paths, relying on a system call that’s only available on one CPU type. And you start to find out that you didn’t even get a lot of benefit from doing that, because you actually locked yourself out of doing anything else in the future. So I think what we have to do is remind ourselves, the reason why some of these platforms work with container images, because those are the most compatible with the majority of the applications in the world, especially those that run on Linux. So just kind of keep that in mind that we’re still talking about fundamental apps that are exactly the same. But we also got to ask ourselves, are we adopting things that are so low level, that means we can never evolve beyond the current machine that we’re using. And that’s kind of what happened during the mainframe era, that people even in 2022, or kind of find themselves locked out of doing anything different. We got two minutes left. And I think there’s some people that are still like, Hey, I’ve heard all of that. I get it. You need platforms to enable people to move fast. So we’re going to end up with maybe Bruno given us a description. Now that people have heard all of that, what is Shipa? Some people think it’s just Kubernetes plus plus, is simply just a dashboard to see existing Kubernetes objects. What is Shipa? And how should they think about that in the context of what we heard today?
Bruno Andrade 47:02
Yeah, the way the way I try to think and explain Shipa is think of Shipa as your application platform, or your application interface on top of Kubernetes. Kubernetes is your infrastructure. So when you think about Kubernetes, if you know refer back is a lot of thinking about what type of hypervisor you’re using what type of ingress like F5, load balancer and all these things, this is great, it has its place. But for you to be able to adopt this in skill at the right place and focus on the right things, you need to be able to bring back application into the main level as the main level citizen in your infrastructure. And Shipa is what brings is that interface that brings that application level on top of Kubernetes, we’re not replacing your ingress is your your cloud of choice. And this is all within your power in your choice, you should choose it. But we allow you to be able to focus again on one or two right interfaces, right? Without having to worry about kind of the underlying choices from an infrastructure perspective. How do you deploy How do you manage and how you’re secure applications, so is bringing that application interface or platform on top of Kubernetes. And by doing this, you can bring DevOps, DevSecOps, SREs, developers and everyone to see the right level of information that matters to them to do their day to day job, right.
Kelsey Hightower 48:16
And look like all technologists, I encourage you to go download it, play with it yourself. answer some of these questions through hands on experience. And I’m sure our panelists who would like to thank today, Bruno from Shipa, and David and Kirk from bamboo HR for hanging out with us today and sharing their stories. We’re gonna catch you all next time.
Bruno Andrade 48:35
Awesome. Thank you very much, folks. Alright, have a good day. Thanks, everyone. Bye
