Accelerate Shifting Left with Value Stream Optimization
Lance Knight: So I'm Lance Knight, and I'm here to tell you that things are changing in the software delivery world. And I think that you all already know the things that are changing software is eating the world today. And software has become ubiquitous. It's in everything. It's in your phone, you have got a computer in your pocket right now, full of software. It's in my washer and it sends me alerts and lets me know that the cycle is dry, it’s done and it drives me nuts. So true, software is everywhere and speed is becoming the new current currency. Speed is really important, or the ability to be ready to deliver software with good velocity is extremely important today because if you're not, your competitors are and to adopt that change, all different types of methods and systems and methodologies are coming out from value stream management to dev ops and agile and shift left and shift, right
For monitoring, all these new things are coming out and it's hard to navigate the waters of the things you want to accomplish. But if you don't adapt and navigate those waters, you're going to become irrelevant. In the United States, right now, a lot of larger banks are doing really well and the smaller credit unions are becoming irrelevant. They cannot provide the same level of mobile and software experience that those banks can. So they're becoming irrelevant and I propose that utilizing value stream optimization can help you navigate these waters and make decisions on which one of these methods, implements or methodologies to implement to help you navigate that water of change. So what is value stream optimization? Well, value stream optimization. Really, you gotta start with describing what's a value stream value stream. Concept's been around since the fifties, started in a Toyota manufacturing production system.
And it's really about measuring your end to end process and seeking value throughout and looking at the methods and things that you're using to deliver software or value to your customer value stream optimization. I define it as the process of looking at the value you're delivering in your software system and making a decision on where within that I need to improve now. Interesting point, when you think about value streams in software delivery, most of those systems have grown organically. They have not been designed, or set up in a value stream way and manufacturing, which I worked in for years, is now and has designed the way they deliver value through their systems. And this is moving its way into software development today. So there's two principles I look at when I want to do a value stream optimization exercise that I've done for a number of companies. Through my years of being in software development, way too many years, I might add. You can look at systems thinking and use systems thinking to drive some change. I'll cover what that is. And you can also use lean principles and both of these are unique and kind of help you take another look at how to optimize what's going on in your value streams.
So let's talk about systems thinking. What is systems thinking? Well, the first thing I have to do is describe what a system is. A system is an input that goes through elements and a phase transition to get a desired output. That's the same in software. I'm going to put my work intake, my requests. It's going to go through a process transition, and you're gonna get an output. Let's use a basic example that everybody can understand. Let's use that dryer software that I talked about earlier. We're not gonna talk about dryers. So in a washer you have a black box kind of system. In the system I put in clothes, I put in electricity, air and out the other end comes dry clothes over a period of time, but it's a little more than that, right? That's just the black box. And I propose that a lot of the IT systems today to leaders and people, it is a black box.
They put in their requests and somewhere down this path outcomes, what they are doing. But there's more to that. Of course we're in software delivery, so there’s more to that. There's more of that in the dryer. So here where I can look at there's a drum, there is a vent. There's a lot more than this. And these here are the elements, the fan, the drum, the heater, all these things go through these elements. I put the clothes in, they go through this and out comes dry clothes, but there's more than that too, because there's feedback loops. So in systems thinking, you're looking to understand all the feedback loops as well. So for example, here, I've got dependencies. I’ve got feedback loops, right? There is a sensor in here that says, “if it's overheating shut off”, and these loops, you look at these through the system, by the way, then when that happens, I then get an alert on my phone today telling me that the system shut off. So, software's everywhere. And those alerts drive me nuts.
So, why did I talk about systems thinking? Because it's another method where you can look at how you're delivering software, analyze it and take some measurements. And with that, you can make decisions. And we take that in and couple that with lean, then we have another approach. Let's talk about lean. So lean is a process. A way of thinking. It has some principles. I learned about lean when I worked in manufacturing as a, younger man. There are five phases to it, or five ways of thinking. Those five ways of course are to identify value, map the value stream, establish a poll approach - I will talk about that in a second - create flow and seek perfection, always be improving and you can use value stream optimization to do that. One of the key things I always like to focus on when I'm talking about lean are looking for areas of what is called “muda” or waste.
These are the eight areas of waste that you look at when you're looking for lean. One of the big ones we always get upon is motion waste. So the process of taking your support tickets and moving them from service now to JIRA takes a few days, that's motion waste, and that can be eliminated today. There's other types of waste, such as over processing waste, over production waste, wait time is waste. And one of the examples I have, I actually look at where we're actually doing motion waste and over-processing waste. And by removing some of that, you're going to be more effective and it allows you to look at how you can remove that through some of those different methods that are coming out today, and different ways of looking at it. Also in lean is thought of creating flow. So my example here is going back to my dryer theme.
If I shoved that full of wet clothes, it's not going to get dry at the same time. And that's the same for your software delivery mechanism. If I just plow a bunch of stuff into there for them to do, it's actually going to slow down. So the theory here is to actually remove work from your system so that you can get more done. I will talk a little bit about that as well. And there's different ways of measuring that. WIP limits is what I'm talking about. Cycle time, lead time, kanban. Combine all these things to help you look at how to maybe implement what you're trying to implement, and implement these things and use it to accomplish your goals along the way. So also in lean there's things that you can use: value stream mapping. We've all heard about that. Now. I think that's been a pretty big year as the SD Times says, “it's been the year of the value stream”, so we can do value stream mapping. This is a sample of a manufacturing one. This is a SIPOC and it means: suppliers, inputs, process, outputs, customers. It's a tool I've used before to define methodologies, or define things in how we do it with our company. And you could use it for software delivery as well. What are the things I need? Who's giving those to me? What's the method I'm going to use to deliver those? Who's going to get it? Customers and so on.
So what I'm proposing and what I have done in the past is I've taken lean thinking and systems thinking, and I use them together to analyze value streams and look for areas of optimization where I can improve flow, maybe by reducing re-work. Remove waste by integrating some stuff and effectively allow you to move faster because speed is the new currency. When you're talking about system delivery, and by the way, speed is a currency. But at the same time, you've got to be more security minded than ever before as well, because those small banks can't see how they're going to open up the flood works of the information that sits within and without exposing a lot of risk. So, they're becoming irrelevant with the larger banks that are figuring that out. They're doing things to make sure security is right. So when I sit down to do a value stream optimization exercise, here are the things that I start with first. What is it I want from this exercise? What are the outcomes I want for the business? What are my goals? Second, I'm going to map the value stream. Then I'm going to analyze all the metrics. All of them, not just the lean metrics, any metric that is out there. I'm going to analyze that. I'm going to ask about how we're doing things. I would start with a why and dig in. Then I'm going to build a future state map, and then we're going to figure out how to execute upon it.
So let's kind of use a real world example. Now I've changed the name to protect the innocent, but I've worked with a company called FinTech. They're a large financial services company in the U.S. They produce credit card, ATM and so on services. And so forth. FinTech has a myriad of problems in their software delivery. First of which, their lead time is off. They have too many tests, too many defects gaps coming back from tests, or from quality into development. The other big problem they had is their flow is off. Their system grew organically, and they've never taken a look at it from an end to end process. So then, I needed to get started with them. So the first thing I did, of course, is what are the outcomes that they're looking for? So I've got to deliver these outcomes, their outcomes, of course, improved quality. They have a big quality issue. They're delivering a lot of defects. Second is reduced rework. That's a very expensive thing in manufacturing, rework, and it's very expensive as well in software delivery, which supports a lot of the shifting left. The reason why shifting left is important is because I want to reduce rework and remove costs by having quality in with development.
Of course they want to reduce costs. Everybody wants to reduce costs in these exercises and improve flow. So they needed to be able to be more effective in it. They were doing yearly releases and that wasn't effective. So what did I do next? Of course, map the value stream. I had a big meeting.
I sat in a meeting room for a day. In and out, came different leaders and stakeholders. And we went through it end to end, understanding everything that happens when they develop software in their organization. And this is an example of that from work intake, to code in production. Way over on this side is work intake. They had clarity PPM. They had JIRA for their help desk. All of this information flows in to get prioritized by their PPM department. Once that's done, they create a work package in word and they hand that off and they re-key that in. And it continues those types of manual processes all the way to the end, where they have a testing product called silk. And I know that's a competitor here, but that's what they had. And there were just all these long periods of time where they would really get stopped right here in test and then move forward. So what I did with that is, I took a look at that, tried to understand that, and I created a value stream map. So this is a value stream map. I put some lean principles in here, and I also put systems thinking in here, not just measuring takt time or measuring cycle time, delivery time and so on. I'm actually delivering and looking at how long those feedback loops take. So, and things started to look pretty interesting as I dug in, for example, 30 days
For something to be in test, 30 days. So, that tells me under delivering too big of a package to test. 20 days waiting on test. Well, why is that? So I also went and looked at all these different metrics. Every metric I could look at, the biggest one that I realized, this is an interesting story, is that the engineering lead came to me and he said, “look, my quality team is working really great. We are finding a lot of defects here, all right, here in our QA team, right before we move that into production, we're being awesome”. Now I looked at the engineering director, and I said, “I think you're looking at this wrong. Really, you're delivering code that can't get through quality”. So what's my problem over here? Yes, the team's doing the well, they're finding defects they’re uncovering them, but we're delivering code that is riddled with defects. So there's a problem here. We have to take a look at it, and then look at the value stream. Of course, there's a couple of things I've found out when I went through this process. First, I found out that the quality department has a weekly meeting where they review all the defects
And then they send them back into JIRA, so that the team can work on them. So, right there's a pretty big delay. I’ve got to wait a whole week to get that defect, put back over into a system where developers can work on it. The second thing I found out is that I have pretty long cycle time through all of these. And because of the long cycle, times, 30 days in test, 20 days to get there, the developers were moving forward in iterative development and coding against that defective code. That's going to come back from quality, thus Increasing all that rework. And just by looking at the value stream, I can start to see what's going on here. So also developers, I found this out, developers are also writing all their own unit tests. They're writing all their own acceptance tests. So the requirement comes in, they write an acceptance test. They know that it's going to pass. It gets through, and it comes all the way back into rework because we're in this isolated model. So I learned quite a bit so that the end result of this is a lot of the principles around shifting left that we've uncovered, were uncovered within this value stream optimization exercise.
We oh, wrong way. And we all know that the further a defect gets down the value stream process from QA to monitor, to code and production. The greater the cost is for repair. So by doing some basic stuff, some basic shift left stuff we've heard. I bet I was able to move left with having developers pair with quality experts to write the unit tests or the acceptance tests. We removed the meeting that stopped defects from coming right back by integrating the two solutions. We also realized that possibly the best thing to do is actually get acceptance tests written and designed by the developers. So they could be better formed, and at the same time, pair a tester and developer together so you could write the code, get it built, run all the tests that they would run on it and move it through quicker. So with value stream optimization exercises like this, this is the type of analysis I can do. I can remove waste. I can implement these processes and I can do a lot more than that, through the method as well. So that's kind of what we understood by this process. I've talked about “what is value stream optimization?”
I put a definition around it. We talked about speed, how important all that is, and using these techniques, you can effectively accelerate shift left with purpose and accomplish some better goals as well. And there's a lot more to this method and process than I can get in 20 minutes or so here, speaking to you. So, just to finish up, I'm Lance Knight. I've been in the IT industry for a long time, and started my career in manufacturing. That's how I know all this stuff. And I also work for ConnectAll. We're an integration company. We integrate tools in your delivery value stream. And if anybody has any questions I'm available. Thank you.
Ready to get started?
Explore the integrations, check out the features, or get in touch