December 10, 2020 at 07:33PM
So often, technology practitioners can feel like the cobblers’ children of tech—we talk about how development and operations can be automated, yet the capabilities we have at our disposal are frequently incomplete, fragmented, complex, and in all a long way from the vision of what tools could be.
So, is there any hope for the future? I spoke recently to Eran Kinsbruner, Chief Evangelist at Perfecto, and Justin Reock, Chief Architect, OpenLogic at Perforce Software, about DevOps and AI, and how the processes of DevOps will be transformed in the next five years by automation. Eran has just got a book out—Advancing Software Quality: Machine Learning and Artificial Intelligence in the Age of DevOps (available from all good outlets), so he should have a few of the answers.
What did I learn? First, the importance of a focus on value in DevOps; second, the role of AI and ML in accelerating DevOps; and third, the opportunities that exist today for AI-based improvement. Our conversation has been edited for clarity, but here’s the key points:
Jon Collins: What does DevOps really mean for you and what makes it work?
Eran Kinsbruner: DevOps is not a closed term that people really understand perfectly. I love Microsoft’s definition of DevOps: it is a union of people, process and product, helping deliver constant value to their customers. Fantastic. But it’s still quite vague.
So how do I do it? I have the people, I have feature teams, I have technology. I’m building these features in a short amount of time. But what is value? How do I know that I’m really adding value to my clients?
Perhaps execution speed is vital to them? In my mind, value is not just about execution speed, it’s much more, but you need to listen to your end users. What are they actually looking to get from your product? Sometimes the end developer doesn’t even know how his feature is going to be utilized out there.
Jon: Yes, I agree—you have to ask: What does value mean to your customers? Suddenly you’ve got a conversation: How do we define value? What are the benefits that our customers are getting? And what are they prepared to put in, in order to get those benefits? It becomes a higher level conversation that can steer everything else.
Without that high-level conversation then you just pump stuff out with no clue. It’s like producing cars. Here’s another one. Here’s another one. Here’s another one. Is anyone driving them? I have no idea! So, and how does that relate to quality in your mind? How does quality play across the lifecycle?
Eran: So I am looking at DevOps from the perspective of end users. Are my end users consuming my products? What do they think about my products? And how can I make sense of all the feedback so I can improve and create more value to these users?
So quality is not just about function: put something in, get something out. You want value equals quality by definition. When you bake into your costs into value delivery, you learn what it really means: functionality, performance, response time, availability.
You discover it by testing what’s right from the end-user perspective because if it’s not something that your clients are dealing with, you’re also not testing or providing quality for what matters. So both from a development and quality assurance perspective, you need to be very focused.
What do I need to cover? What do I need to test? On which platform? Which scenarios? Which is the most eloquent feature that someone actually touched in the previous code commit and stuff like that. This is when you find the result: valuable features or products to your end users.
Jon: Great, let’s make it value-first. But how does this map onto the DevOps process, from a pipeline perspective? And how can AI help?
Justin Reock: When I think about DevOps, it goes back to the Theory of Constraints and applying the idea of reducing the amount of friction involved in converting value to throughput. That to me is the essence of DevOps, at least from a business perspective. We’re doing everything we can to reduce the amount of “laying around” inventory, i.e. code that has not been converted into money yet.
The more we can do to reduce friction between converting our inventory and organizational costs throughput, the quicker every line of code that a developer commits to a source control repository becomes throughput, or money, out in the market. And if you distill it back to that birth, then I think that if you look at AI, its place becomes very clear.
The ideal DevOps pipeline is one that will be completely frictionless: a developer checks in his code and that code is then running in production five seconds later, right after passing through a series of tests where no human was ever involved. The customer is buying something, and you converted that code into throughput in a matter of seconds. That’s brilliant and beautiful and elegant, and that is the goal of DevOps and software, and so AI.
Jon: Let’s get down to the nitty gritty—can we look at an example?
Justin: Sure, for example software testing? There are multiple points where we can remove not only the slowdowns that having humans as part of that process adds, but we also, if we do it right, can eliminate more and more tester bias from that pipeline, which means we have less and less retesting. In a lot of ways, we’re still brute forcing the way we deal with that problem. We do A/B testing and Canary releases, just in case we didn’t think about a possible pathway.
But we still have goals here: DevOps is all about the continuous feedback loop. So you have to get feedback about your product and you have to integrate that into new features and you have to fix bugs, of course. The more we can reduce those issues and prevent them from seeing the light of day, through things like fuzzing and AI, the faster we can get that code out making money.
This all ties together. In a world where it’s all connected software, it opens the door to ambient services, self-driving cars, or completely automated retail venues. It helps create our fully realized virtual and augmented reality where everything is a digital asset, and scarcity is proven through blockchain, but that blockchain only matters if quality is enforced.
Jon: Whoa. That’s quite a leap!
Justin: Yes, you’re right, but I don’t think people really understand the molecular level at which software is about to bloom, due to AI in the DevOps process. Reducing friction in the pipeline is the biggest necessity, and it will open up all kinds of opportunities.
Jon: OK great, let’s drill into this –what is the lowest hanging fruit? What is going to change in DevOps over the next few years, because of AI and ML?
Eran: Let’s think back to the feedback loops. Sometimes developers and DevOps managers think they got it right, and are doing things right, but then a machine learning algorithm comes and sets them free, providing feedback which is quite different from what they thought they would get. ML can help provide unbiased, objective feedback, which doesn’t really look at the product roadmap or anything like that, but it looks at the end users, which is kind of clean.
Then when you merge it with the product decisions and the software delivery cycle, maybe you’re going to get something more solid and more relevant to your clients. That’s what I see as the biggest opportunity right now.
Jon: That all sounds great, it’s great theory. But what do I do to address these ideas?
Eran: That’s a good question. You don’t need to throw everything away and AI cannot really solve everything immediately. But we do need this acceleration of software quality. The noise reduction, the prioritization. We can obviously apply them throughout the entire pipeline, but let’s just focus on testing.
The test cases that are the most unreliable are a good case in point. We call them flaky. They’re showing up red in your CI/CD pipeline and you’re doing nothing about them because you don’t know why they are failing. AI is able to look into these failures, and classify them into different buckets. And suddenly we can see 80% of all these failures are not real bugs. They’re just down to poor coding skills by a test engineer. We now can zoom into the 20% that are real bugs, that are real issues that may impact the value to my customers. Now I have something I can prioritize. I know where my developers need to focus.
So noise reduction and prioritization of testing can result in an acceleration of software delivery. Once you’re applying that into your existing processes, you can move much faster.
Jon: Great, thank you! So AI and ML may unlock huge value in an increasingly digital world. Key right now is to look for direct opportunities to remove friction from the process itself, in testing and across the pipeline. Eran and Justin, thank you very much for your time!
No comments:
Post a Comment