In our previous posts, we have looked at some of the principles that apply to DevOps and how we can use them to guide us during our company’s transition to DevOps.
In this post, I’d like to take a step back and review how, from my point of view, Lean thinking has greatly influenced the emergence and development of the DevOps movement.
I do not intend to provide a detailed description of what Lean thinking is, since there is a lot of literature on this subject. But I do want to give my vision of how the same principles that revolutionised the manufacturing sector have also revolutionised the software development industry, and how DevOps is based on those principles.
In this respect, it is important to remember that when the Lean movement emerged, thanks to Toyota's Taiichi Ohno, the manufacturing model was based on mass production, which was considered the only way to produce cars at low cost.
The Japanese market's inability to follow this strategy made it possible for the Lean movement to emerge. In other words, if the Japanese had been able to follow the dominant strategy, they would certainly have done so and would not have had to devise a new competitive strategy. Therefore, this competitive disadvantage meant they had to come up with new ways of working in order to compete, and these eventually became the established strategy in the manufacturing sector.
Similarly, in the field of software development, the crisis with the traditional model has led to the development of new, much more competitive strategies. If, using traditional methods, we had been able to deliver that value, then surely the DevOps movement would not have had such an impact. But the reality is that, since it is impossible to develop software and deliver value at the speed and quality demanded by the market following "traditional development methods", the DevOps approach is currently establishing itself as the strategy for success when it comes to delivering value to the customer.
Following this reflection on Lean thinking, I will now briefly describe the principles that made this revolution possible and discuss its influence on the DevOps movement.
Principles of Lean thinking
Lean thinking is a very wide-ranging topic and it is far from my intention to provide a thorough description of its principles, but it is important to know what they are. I will focus on the principles that apply to software development as defined in Mary and Tom Poppendieck’s book entitled Lean Software Development: An Agile Toolkit – which I do recommend you read-
The principles of Lean thinking are as follows:
- Eliminate waste. This is the main principle and the one that first comes to mind when talking about Lean. Only by eliminating waste does the improvement in productivity become really noticeable. The important thing is to determine what is waste and what is not. On this point, a very useful tool for detecting waste is the Stream Value Map, which anyone practising DevOps should use. This tool visually maps all the activities that are performed to carry out a specific job throughout the entire flow of delivering value. For each stage, we identify those activities that provide value, those that have longer waiting times, those that generate inventory, and so on. In this way, we can identify bottlenecks and areas for improvement. In upcoming posts, I will describe ways to apply this in order to reveal this waste.
- Amplify learning. Lean thinking is based on continuous improvement and creating a culture of continuous learning. Here, the relationship with the DevOps movement is more than evident because this is one of its fundamental pillars.
- Deliver as quickly as possible. This point links directly to “The First Way to DevOps” – accelerating the flow. This principle gives rise to all the practices that try to reduce cycle times.
- Decide as late as possible. This principle is closely related to the previous one. If I can deliver very quickly, I can afford to wait until I have as much data as possible and thus avoid making decisions based on incomplete or erroneous information. What I do need to be able to do is ensure that, once the decision has been made, delivering it is done quickly.
- Empower the team. The idea of agile, self-organising teams is rooted in this principle and is one of the basic pillars of DevOps, as we saw in the previous post.
- Build integrity in. This principle is directly linked to incremental design, refactoring, testing and quality assurance practices. Lean thinking is very much focused on quality and not letting mistakes slip through the value chain. The deployment pipeline concept is based directly on this principle.
- Consider the system as a whole. Last but not least, Lean thinking considers value delivery as a complete system, avoiding local optimisations that are a significant source of desynchronisation and waste.
These are, very briefly, the principles of Lean thinking, and we can clearly see their influence on DevOps. However, I would like to share some (personal) reflections on certain points of particular interest in this comparison.
Lean and DevOps
If we compare the principles of Lean thinking with the three principles of DevOps that we condensed in “The Three Ways to DevOps” (accelerating the flow, amplifying feedback and creating a culture of continuous experimentation and learning), we see that they have a direct translation.
As far as the "eliminate waste" principle is concerned, the key point is to look at what is considered waste from the perspective of software development in general and for DevOps in particular. Waste will be anything that does not provide value and, for DevOps in particular, this includes all wastes described in books on Lean, such as excess inventory in the form of partially finished work, bugs, and so on. One of the specific types of waste is related to the delivery of work packages, which in manufacturing is very clear, but in a DevOps environment has a specific meaning. In this environment, it relates to the movement of work between teams. In other words, every time work is passed from one team to another, a type of waste related to transport or motion is being incurred. In DevOps, when Dev and Ops work together, this waste disappears. We would be surprised at the amount of waste generated from this separation.
Another type of waste that results from the separation of departments is waiting time. All the time spent waiting in a queue is time wasted. With the use of DevOps, this time is again shortened, because the waiting time between Dev and Ops is reduced to virtually zero. This, combined with empowering the team when making decisions, means that waiting time generated by complicated change committees or other bureaucratic processes is drastically reduced.
From my perspective, only the reduction in these two types of waste (motion and waiting) justifies the improvement in cycle times, as can be seen when using the Value Stream Mapping tool.
Finally, the fundamental shift in thinking is that of “Consider[ing] the system as a whole”. It is fundamental because of the enormous impact it has and how difficult it is to accomplish.
Realising that the process of delivering value in a system is a holistic process, in which the whole is greater than the sum of its parts, involves a major change in strategy to achieve the goal of delivering value as quickly as possible.
The difficulty in the change in mentality stems from the fact that, to tackle a major problem, traditional teaching says that the problem must be divided into smaller parts, so that each of them can be optimised separately, thereby optimising the entire process. This way of thinking inevitably leads to local optimisations within the process that in turn end up generating efficiency losses in the process as a whole.
This is what has been happening as a result of the idea that it is possible to separate development from operations, optimise both parts separately, and manage to increase the overall performance of the system. It has been shown that, far from improving the system, it actually makes it worse.
A very popular comparison to illustrate this idea is preparing for a marathon. To win the marathon, optimising every kilometre is not a good strategy, because this would mean sprinting every kilometre and ultimately losing the race. What is important is the whole thing, not the local optimisation. The same is true of DevOps. Its greatest strength lies specifically in optimising the process as a whole, and that is why DevOps is really more a transformation of an entire company at all levels rather than an optimisation of one process or another.
The important thing is to realise that the value delivery process is a complex system with non-linear components. These non-linear components are the people who form part of the process.
Once we have determined that we are faced with the difficult task of controlling a complex system, trying to apply the wrong control techniques to it will lead us to failure.
Lean thinking and DevOps take this into account and hence their effectiveness.
" DevOps’ greatest strength lies in optimising the process as a whole, and that is why DevOps is really more a transformation of an entire company at all levels rather than an optimisation of one process or another"
To summarise, in this post I wanted to reflect the huge influence that Lean thinking has on the DevOps movement to help you understand the reasons behind many of the strategies that are applied to DevOps and why they are really effective.
In upcoming posts, we will take a look at some of these strategies and see how they align with the principles of Lean thinking that I have briefly described here.
If you found this post interesting, don’t miss out on others in the series. Subscribe to the blog now.
Emiliano Sutil is a Project Manager at Xeridia
