Kubernetes: maturity, challenges and the impact of AI (Shaun O'Meara, Mirantis)

Speaker 1: 0:23

This is Sander. I'm at KubeCon Europe. Just briefly, first impression. I mean lots to do about AI, obviously at this KubeCon. Yeah, and just all the hype now, yeah, just to get it a little bit out of the way in this discussion. So what do you make of all this and what's the impact of AI on Kubernetes or on containers or in cloud-native development in general?

Speaker 2: 0:47

I think the biggest impact that we're seeing when it comes to AI is usage. I mean, ultimately, more and more people are trying to work out how they're going to bring AI into their infrastructures. Ai is impacting. It's just another workload. Let's just be very blunt about it. That's ultimately just another workload. It does, but it is a fundamentally different workload to Different parts of it. For sure. I mean, if we're starting to talk about running ML or large language models, yes, we've got to change the way we think about infrastructure, but from a developer's point of view, once that infrastructure is up and running, and we're talking about inference engines or access to GPUs, once that's been solved and, yes, it fundamentally does change a lot of the networking infrastructure we need once that's been solved.

Speaker 2: 1:30

From a developer's point of view, it's just another set of microservices applications running that talk to. Hopefully, if you've done it right, talk to an api that provides you access to the language model or those components.

Speaker 1: 1:42

So all the, all the hype or the buzz or whatever you want to call it nowadays at a conference like this and all the others, how deserved is it?

Speaker 2: 1:54

I'm going to get myself into trouble here. No, you don't have to. No, the reality is I think there's a lot of deserved hype. What large language models have done is they've changed the way that humans can interact with machines. I mean, I keep saying and I don't want to offend anybody, but it allows your grandmother to talk to the machine. And in a world where, frankly, kubernetes is still highly complicated for many people, if we can simplify that by providing interfaces through large language models, that's great. But LLMs are not all.

Speaker 1: 2:27

Ai is oh, and ai has been around from way way longer than then, and llms have been.

Speaker 2: 2:32

I was reading about a test case in the 1950s. You know, people don't think computers existed, but you know, hell they did and they were being used and there was some amazing research work done. But the reality is, what's most interesting about this for me is all the stuff that's coming up around the open source ecosystem. Yeah, you know, llms are cool. Yeah, uh, you know, open ai has done an amazing job of really getting everybody access to ai technologies.

Speaker 2: 2:57

But what's really cool is all the open source stuff that's happening um you know, lang chain, um, all the vector stores, the really cool research going into embedding that stuff is really what's going to make the difference and that's what we start to be able to implement AI in our operational side of running these infrastructures.

Speaker 1: 3:17

Yeah, but then you're talking about the impact of AI on running your infrastructures and developing the tools, tools that you use to do that.

Speaker 2: 3:28

That's one area of impact. Yeah, I think that's really an important area, especially for a lot of the community we're talking I think so too.

Speaker 1: 3:33

I mean that's the most important one. I mean the the other side, I mean, and I think that's it gets diluted a little bit by all the focus on gen ai, obviously, which is which is more of an assistant kind of thing, and not necessarily an assistant, but more summarizing a lot of stuff and it helps you understand stuff better. That's more of the final application that you put AI, gen AI in, and I think the more interesting cases are using AI to actually make better applications.

Speaker 2: 4:06

I get where you're going. I mean, I'd slightly disagree with you. I think the human machine interface component that AI brings is incredibly interesting because now to use an overused term we're kind of democratizing access to infrastructure Not just infrastructure, not just infrastructure, but into IT systems. When a normal, non-developer person can now start to create tools to interact with data in a way that's more natural for human beings, using voice and word and text, that becomes very, very interesting for accessibility. Now I don't disagree with you that what the really cool engineering use cases very much are happening in.

Speaker 2: 4:52

How do we manage our infrastructure when it comes to using this new AI tool, especially when we start looking at you know, just look at Kubernetes we get thousands and thousands of random log messages. What many people would look at as an error is just normal. It's the nature of communities, yeah, but if I can start to log all those things, use these AI tools to find patterns in all these things, now I can start to say, okay, I'm running an application, I can identify and correlate those patterns across that application. That's going to be able to give me things like well optimizations. You that's going to be able to give me things like well optimizations Instead of random example, but instead of doing 20 calls to an API, I can do one. I can see what happens when I do 20 calls across the system and that's really important.

Speaker 1: 5:37

I mean that's also necessary because it makes things faster and more efficient and that's always good. It's still more efficient. Lower latency, yeah, especially when we're moving into an era where you need more and more to squeeze more and more out of your resources anyway.

Speaker 2: 5:52

But also an era where, when you have front-end applications and you have an audience that's expecting a certain level of performance and you click on a directory search and it takes 20 minutes to come up. Nobody wants that, no. So if you can find ways to at least get the perceived experience of your application to be considerably faster and more responsive, now you're changing your audience.

Speaker 1: 6:20

Yeah, and in a world where we are splitting hairs in many application spaces. That's really, really important. Yeah, yeah, I, I agree with you. It's just uh, how, how would you, how would you characterize where we are in this, in this, in, in this, in all this trajectory?

Speaker 2: 6:36

Depending on who you talk to. I mean, we're really at the top of the hype curve right now. Um, I think we're going to, we're going to run into some people are going to suddenly realize that this is just hype and that it's not really production ready in many use cases. There's going to be a lot of challenges there. We're seeing some of those challenges. Everyone right now is very focused on the public API, chat, GPTs and the like. Reality is not every company is going to be able to use or allowed to or want to take the risk.

Speaker 2: 7:10

We're in Europe, we have great privacy protections as residents of European countries, but we expect the companies that have our data to take care of that data, and I don't want them, frankly, I don't want them being pushed into OpenAI.

Speaker 1: 7:28

No, I think that's a fair desire Not to want that it might be too late, but yeah, oh well, for some things it is. But yeah, you see some interesting initiatives coming up in that respect. Right, and to be honest, most of the models I think, from where I'm standing, most of the models that you most of the load is going to be on inferencing and that's going to be local anyway. Yeah, you're not going to do that in the cloud. So maybe you will, but I mean, but there are many good reasons to think of doing it not in the cloud and it's possible at least. But with training, for example, that's just going to happen in cloud.

Speaker 2: 8:13

Yeah, it's virtually impossible and I don't think it's going to be your traditional cloud. I think we're going to start to see more and more, and we are. I mean, just look at what NVIDIA is doing, look at what the big cloud providers are doing More and more really specialized training that are going to differentiate based on how fast you can train on those infrastructures. But I agree with you, inference is going to be done local, and I say that in inverted commas because it might be running on a GPU provided by a hyperscaler. Yeah, you can always have the local cloud or whatever. Yeah, but it won't be utilizing a inference service from those you know with their models behind it. Won't be allowed to. Won't be allowed to.

Speaker 1: 8:55

But is AI also the primary kind of challenge or the thing for the community itself to actually tackle at the moment, or are there other things that you see happening that are maybe even more important or slightly overlooked, maybe?

Speaker 2: 9:12

Yeah, I mean look from a community. Right now, communities are still hard. We still don't have standards around things like serverless containers. You know we have every hyperscaler. Every vendor has their own serverless container model. What that means is in a world, again in Europe, where we're having to deal with registration, like DORA, where portability is really important. It's just one example. We need to find a way to create common standards and our community has a habit of repeating itself and not sticking to standards and recreating the wheel, and if we can find better ways within the community to prevent that from happening, that'll go a long way to people trusting open source even more than they do today. And at the end of the day, the commercialization of open source is all about trust. It's about creating trust.

Speaker 1: 10:03

Yeah, so there's also this thing going on at the moment called the platform engineering right.

Speaker 2: 10:14

Yeah, it's a big topic as well. I always smile when people talk to me about it.

Speaker 1: 10:18

Yeah, well, I'm glad I made you smile, but there's a lot of maybe even also hype again around it, but some of it, I think, makes sense Absolutely. I mean, especially when you referred to it earlier, kubernetes is still very complex, so if you can somehow move into some sort of platform engineering approach, that would make it less complex at least, I mean not the foundation, but you don't need to worry about that. You don't need to worry about that. That's the point, right. I mean not the foundations, but you don't need to worry about that. You don't need to worry about that. I agree, that's the point right.

Speaker 2: 10:48

I mean absolutely. I think what we have to be careful of is over-defining a platform that doesn't meet the needs of an organization. And what is a platform? At the end of the day, it's a set of tools necessary to run my applications and my applications, so my unique applications, in a production environment consistently, because Kubernetes on its own is just an orchestration layer and to all the Kubernetes. People out there don't shoot me for saying that, but ultimately what makes it truly valuable is what I can add onto it to support my application and to accelerate the development of applications. But there's another side of it which is important when we talk platform, and that's providing guardrails to developers. It's providing developers, helping them abstract the complexity and provide them the security standards, the rules and the tools so that they don't have to go and reinvent the wheel every time they're doing something. That is what a platform is to me, but we need composable platforms to achieve those goals platforms to achieve those goals.

Speaker 1: 11:54

Yeah, yeah, somebody came in, but yeah, and we just wanted to wait until he slammed the door, but he didn't, so we could have just gone on. Oh, that's fine, no, so, so, so. So, platform engineering is real, right, I mean, and it, and it makes sense. But what does it say, what does it do for the maturity of Kubernetes in general? Because there's a big. Yesterday, during the keynote, there was this Linux moment. People are talking about Kubernetes reaching its Linux moment and being at a point where it was relatively mature after 10 years.

Speaker 2: 12:32

I think Kubernetes itself? Yes, certainly. I think the delivery of platforms on top of Kubernetes in a consistent way is not there. There are too many different options. There's no consistent standard. Look, helm's great, but great in one cluster. You're still composing things with Helm. It's still highly complicated. Is the Linux moment there quite? I don't know, I wouldn't think we've quite hit the full Linux moment, where it's ubiquitous and standard. That said, linux isn't particularly easy to use either. That's true. I mean, I've been working with it for probably 50 years and I still don't find it particularly easy. No, that's true. But yeah, maybe I'm just not smart enough.

Speaker 1: 13:19

I don't know who knows I mean if you became CTO, so I would hope you're smart enough to be able to use it.

Speaker 2: 13:29

Promote you to a level of incompetence.

Speaker 1: 13:31

Yeah, the Peter Principle or whatever it's called. Yeah, so, but from a standpoint, from your standpoint, right from Mirantis' company, what does it mean? That we are moving, albeit maybe slowly, into a platform engineering kind of world? Does it impact your not necessarily your business, but your customers?

Speaker 2: 13:56

How does it impact you? It certainly impacts our customers. I mean, which is our business at the end of?

Speaker 1: 13:59

the day. I mean there's not a big separation between those two things.

Speaker 2: 14:04

We're in the business of supporting enterprises to reach their goals. I know it sounds a very vague thing for a CTO to say, but at the end of the day, what is our job? We have to make it easy for customers to produce valuable code that's for their business. So the whole platform engineering approach right now we take the approach we're going to put Kubernetes everywhere and create a baseline fabric, but that's just a starting point. As I said earlier, kubernetes on its own doesn't really give you much.

Speaker 2: 14:36

So what platform engineering and the platform engineering approach means is that the audience I'm talking to when I'm delivering my product is the platform engineering teams. I don't need to go and create a one-fits-everybody or one-size-fits-nobody you know in socks approach, because those platform teams are going to take responsibility for composing the components they need to service their audience. What I need to do as a business is give them the tools to make that easy, consistent and again I'll come back to this term safe. Ultimately, they need to know that somebody's got their back. I'm doing it with open source. I'm not trying to reinvent the wheel anywhere, but I'm trying to give them that safety net that they can come in and deliver on.

Speaker 1: 15:22

Yeah, I know we've talked about this before on different occasions, not necessarily today or on this podcast but there is this thing I mean, when you start thinking like this for customers, right, and giving them a platform that's safe and you know all those components, you get into sort of an opinionated kind of stack, which is, I mean, I'm all for it because I think that makes sense. But where do you see the balance in this opinionation?

Speaker 2: 15:54

I think it's fine to be opinionated, I think you need to understand who your audience is. Who are you talking to and who are you providing capabilities to. We're trying to walk the line between being fully opinionated and composable. Walk the line between being fully opinionated and composable, and the way we do that is by really focusing on a truly curated set of mature projects within the community. Where we need to, we're going to really heavily contribute back to those projects to help them with that maturity and that standardization. Provide solutions. So life cycle management is really where our core skills are. So provide lifecycle management solutions that allows platform teams to compose what they need out of that curated set. Help them work out how to curate the extra pieces and then do it for a small number of unique organizations out there.

Speaker 2: 16:50

I'm not going to try and create a one-size-fits-all. No, I'm going to try and create a platform that allows my platform operations team to create what they need. Yeah, so my opinions are inserted into the lifecycle management tools. My opinions are inserted into what I think is mature in the industry, and maturity is everything about. How big is the community? How many code check-ins have there been? Are they meeting the use case? And those are the things that I count, that I think count.

Speaker 1: 17:23

Yeah, but that's I mean, I think that makes perfect sense for that, but how does that? What does it mean for your target audience in terms of the companies that you target? Is this approach suitable for everyone? I mean, I'm just trying to, yeah.

Speaker 2: 17:39

You know, we see the world that there's a subset of customers out there who want to have control of their digital destiny I'm rolling over my own tongue here, yeah well, it's almost five, it's been a long day. But there's a subset of organizations in the world that really want to focus on being in control of how they build their infrastructure going forward and by infrastructure I'm using it very broadly here and they don't want to be locked into a certain vendor. We see that with some of the changes happening in the industry today. You know, suddenly people are being told well, I'm increasing your price 10x.

Speaker 2: 18:20

Vmware. Vmware, for example, the Broadcom components, or you know they're end of life-ing a product, but not really end of life-ing a product. So how do we give customers the optionality not to be locked into that type of model yet still give that safety and support? And that's where we see those customers who recognize that they need control of their destiny, are willing to take control of their destiny. And that's a small but significant subset of organizations in the world and we have quite a lot of those in our customer base today.

Speaker 1: 18:57

Going back to reconsidering after a sort of a 10x increase of licensing. Do you see a lot of VM replacement nowadays? Probably for your customers, yes, certainly for our customers, yeah, but there's also a big chunk of customers that are wall-to-wall VMware. They're probably not going to do this anytime soon, right? I?

Speaker 2: 19:17

think the VMware world can be separated into a number of different buckets. I mean, you've got the big, large-scale VMware customers who are using all the cloud capabilities and functionality of VMware. They're a very particular type of customer. Actually. They're quite forward-leaning in many ways because they've taken advantage of these cloud capabilities. I don't see them shifting anytime soon. But in between them and the very small customers, there are enough customers there who are looking for, if not alternatives to VMware, complements to VMware. They want to move a significant portion of workload to another vendor or another platform that can give them negotiating power and optionality and that ability to say hey, I've still got certain workloads on VMware because they're certified there, they work there, the vendors support them there and it's great.

Speaker 2: 20:09

Vmware does very well and has done very well. Fantastic technologies, yeah, but there are enough workloads out there that don't need VMware underneath the hood. So that's one approach, vm to VM. But I think the biggest opportunity right now is a modernization opportunity and if we can find ways to work with customers to do a net zero or largely net zero migration so they can keep their costs flat over time. Or maybe some minor bumps here and there. We're going to keep that careful, but take those vms and get them into containers. Now we're changing the ecosystem, but they still use vms. They still use vms. It's on a different platform, that's one. Or we actually take those VMs and get them into containers. Actually modernize those, yeah.

Speaker 1: 20:59

And does it work to have your VMs in containers? Because back in the day, a long time ago, when a cloud came up, they said, well, lift and shift, don't do that. Is there a similar argument to be made for VMs in containers of it, or is it a different?

Speaker 2: 21:17

discussion. It's a slightly different discussion. It's not the look. Many people lift and shifted into cloud, yeah, and they were not happy. And they were not happy. They've been burnt. It cost them four times as much because they didn't change to cloud patterns.

Speaker 2: 21:30

You can make the same mistake with VMs to containers, and many will, but the reality is, if you're careful and you're smart about analyzing the type of application that's running and you do that shift from that application into a container in a controlled way and you pick the right workloads and there are enough of them out there then you can do a straight migration. Workloads and there are enough of them out there then you can do a straight migration. Many applications are going to need to be refactored. It's just reality. So you have to then do a cost analysis Is it worth my while refactoring or do I just continue to run them in VMs Because it ain't broke? It works? The real question, though, comes as you look at the curve of what's it going to cost you to maintain the legacy over time? We all know that legacy costs go up.

Speaker 1: 22:15

Yeah that, and I would imagine you would also like to accelerate your own development and your own company's development, right?

Speaker 2: 22:25

Yeah, your own company's value in its IT. So why do we Then all this?

Speaker 1: 22:30

legacy may. Yeah. Well, that's going to hamper you at least in the future, if not already now. But this is a hard decision to make, I would imagine.

Speaker 2: 22:41

It is a hard decision to make, but I mean, there are great tools becoming available on the market. There are great small companies that can come in and help you do that analysis, great small companies that can come in to help you do that analysis. But it's an analysis of cost versus value to your business, versus, you know, are you just putting a Band-Aid over it and long-term you're going to have to excise the wound. You have to make that decision and it's one that needs to be done.

Speaker 1: 23:13

And as we've been talking about the future, how do you see the future? Take out your crystal ball and look a year or two to the next cube columns here in Europe.

Speaker 2: 23:26

So look the crystal ball gazing is always a tough one.

Speaker 1: 23:32

That's why it's so nice if you can do it.

Speaker 2: 23:35

I think more and more, we're going to start to see consolidation. Still, if I go and walk the floor, there are so many different networking options, so many different security options, but we're slowly seeing that consolidation start to happen. So, from an industry point of view, I think Kubernetes is going to get more stable, more, more There'll be fewer people doing large scale multi-cluster deployments. There'll be some winners there. Simple, there'll be some consolidation there. Networking side of things, there's always cool new little networking companies starting up. But I'm going to think what we're going to see is we're going to see more standardization there. Bluntly, crystal ball multicloud is going to be the norm going forward. It won't be what it is today, where it's only the cool kids are doing it. It's just going to be normal.

Speaker 1: 24:27

Is Kubernetes ready for that?

Speaker 2: 24:30

Kubernetes itself. Yes, we've got to kind of get out of our own way in that case, but then that we get to the previous discussion again. Itself, yes, but all the stuff, the ecosystem around it is not quite ready for, and I'm not sure that the users are ready for the complexity way you know, 30 years ago Linux was complicated and this beast, linux, hasn't changed that much. But we've learned, we've grown.

Speaker 1: 24:58

Well, yeah, it is, yeah, you grow together with the technology.

Speaker 2: 25:03

And it starts to get less complicated and more standardized and more supportable and, you know, dealing less with bugs and more with trying to create. So that's definitely an area I would say is the future, this idea of Kubernetes as a fabric across everything. We're going to see more alternate technologies. We've all spoken about Wasm at the link.

Speaker 1: 25:26

Yeah, it's funny. By the way, I haven't heard many people talk about this yet, but ubiquitous computers is going to be a real thing.

Speaker 2: 25:33

Yeah, I haven't heard many people talk about this yet. No, but ubiquitous computers is going to be a real thing. I still feel strongly that we're going to see more mobile devices with more ubiquitous compute-type capabilities, with GPUs on the edge providing for a lot of everyday services. We've been talking about it for 10 years All the 3G, 4g, 5g, 6g in the telco industries and on. We're actually at a point now where it's possible and could become. Well, it's about time, then. Is the investment there? And what is the killer application for those things? That's really it, and, of course, I believe strongly we're going to see a bit of a slump in the AI hype, but then in the next two to three years, ai will just become I had the same impression.

Speaker 1: 26:29

I recently was at an event by HP and they were talking about their AI PCs. You know that stuff, I talk about AI at the edge and I was like, okay, but I mean, and they were saying a five-year-old, five-year-old there are 10 million five-year-old or older devices out there that can be replaced. But that doesn't mean there's no reason. The reason they replace it with an AI PC is not because there's AI in that PC, it's just because it's the next generation. It becomes interesting from that perspective if they change after two years. I want this AI PC. I don't see that happening and I think that's what you're alluding to as well. It's just going to be the next step. I think that goes back to.

Speaker 2: 27:20

It'll just be part of everyday life. It'll be within everything. You know our phones my phone can run AI right now yes, particularly fast, but it's there. But the reality is a lot of that stuff is going to be run through API services on a back end. Yeah, because, frankly, we have the connectivity for it. We don't need to run everything, but the user interface and the processing. If we could start to push it down to the edge, the real impact of doing that is sustainability. I think yeah, and if I'm not running everything in big fat data centers everywhere and I can run it on the edge devices and push that you know, wiki, team cooling, etc.

Speaker 2: 27:57

Requirements to somewhere else, I change a lot even if AI is just another workload even if AI is just another workload, I strongly believe it is the inference side of it and the GPU side of it is we're going to see a big change in that world and it becomes a scale step change.

Speaker 1: 28:17

So we ended up with AI again. So, even though we said we were going to talk about it, and we talked about it for, I think, for about 10-15 minutes of the 30 minutes that we had, but I think it's a good point to close. Thanks for talking to me and I look forward to speaking with you again in the future. Likewise, thank you very much.