In our last post, we described the features that the tools we choose on our in our organisation must have.
In this post, we want to share with you the aspects we have taken into consideration in choosing the tools that are part of Orbis, Xeridia’s DevOps platform.
To select the most appropriate tools for each environment, aspects such as the following will need to be borne in mind: integration capability, support for agile methodologies, ease of implementing deployment patterns (deployment pipeline) in our system, ease of delivering customer feedback to development teams, and the facility that they offer for encouraging a culture of continual experimentation and learning to emerge within the organisation.
INTEGRATION CAPABILITY
The first point that needs to be resolved is whether it is a good idea to have tools from different manufacturers or whether it is better to use a suite of products from a single manufacturer. Everything has its pros and cons, but at this point we will try to answer the basic question: Which answer better promotes collaboration and communication between teams?
In this respect, having integrated tools that centralise and exchange information in real time, between team members, customers and users, seems to be the appropriate response.
If we opt for products from different vendors, we must give serious consideration to the integration capability between all of them. We must make sure that these products provide connectors to products from (at least) the top manufacturers or an API to integrate these tools with third-party products.
SUPPORT FOR AGILE METHODOLOGIES
Another important aspect to consider is that the project management platform chosen should support agile methodologies (such as SCRUM, Kanban and LEAN). Therefore, at this point we should ask ourselves the following:
- Does the tool allow me to create prioritised product backlogs?
- Can I set Sprints based on the items in that product backlog?
- Does the tool allow me to display the work on boards?
- Can I set limits on work in progress (WIP)?
- Does it generate Burn Down Charts for me?
- Does the tool allow me to perform Value Stream Mapping?
- Can it easily calculate the cycle time of a system?
- Does it allow me, through analysis, to detect system bottlenecks?
In other words, is the tool geared towards agilism and reducing cycle times?
For example, we could say that a spreadsheet has all these ingredients. However, what is better in terms of promoting collaboration between teams, a spreadsheet or a shared work board in a web-based environment? Personally, I am of the opinion that spreadsheets and documents are very useful for a person who works alone. However, they are not so useful for teamwork, where versioning and updates to information complicate matters.
Would a tool that specialises in producing Gantt charts be appropriate? Unlikely, as with such a tool you cannot answer “yes” to the previous questions. It would certainly be very useful in other areas, but not for implementing DevOps.
“Having integrated tools that centralise and exchange information in real time, between team members, customers and users, promotes collaboration and communication between teams.“
At Xeridia, we chose Jira Software. This tool can fully support the SCRUM framework by creating prioritised product backlogs, Sprint backlogs, Kanban boards, WIP limits, automatically producing Burn Down Charts, generating accumulated flow diagrams and process control charts, etc.
Furthermore, combining this with Confluence allows us in turn to accumulate knowledge and make it available to anyone who needs it. The Jira-Confluence pairing becomes into a very powerful tool for project management and collaboration between all those involved. Confluence provides many options for promoting teamwork.
DEPLOYMENT PIPELINE
As we saw in the previous post, another key factor is that the tools we select must contribute to reducing the cycle time. In this respect, it is essential that the tools can be integrated to build what is called the Deployment Pipeline.
A pipeline would be the sequence of required tasks or processes through which work passes, from its conception as an idea until it is delivered to the customer.
More specifically, the deployment pipeline would be the sequence of tasks executed from the time the code is uploaded to the version control server (Commit Stage) until it is successfully deployed to production and being served to users.
Generically, the tools required to implement a deployment pipeline would be:
- Version control server.
- Integration/continuous delivery server.
- Storage server for artefacts generated during the construction phase.
- Testing frameworks that serve as controls for moving from one phase to another. This also includes code quality analysis tools.
- Automatic deployment mechanism.
The most important point is that you can view the pipeline at any given time to see its status.
Expanding this a little further, within each type of tool for implementing the deployment pipeline there are myriad possibilities and combinations. Let’s take a look at some of the tools that Xeridia recommends:
Firstly, in relation to the version control server you could choose Subversion, Git or Bitbucket, among others.
Continuing with the idea of maximum product integration, within its solution, Xeridia suggests using Bitbucket as the version control server and Bamboo as the integration and continuous deployment server.
Xeridia selected Artifactory as the server for storing the artefacts generated, although others are available, such as Nexus.
As regard the testing frameworks, it is worth remembering that what really gives the pipeline consistency is the test suites used at all levels: unit, integration, system and acceptance testing. We can only achieve continuous delivery when we have a suite of tests that gives the system the confidence to certify that code can be promoted to production automatically. We must never forget that the quality of the code should be a factor that should limit its progress through the pipeline.
The variety of implementations in this area is very diverse and dependent on the projects and technologies concerned. Xeridia has opted to use TDD and BDD methodologies to automate testing at all levels. The use of BDD follows the principle of maximising the flow of communication by establishing a common language between users and developers that facilitates communication and understanding between these two groups participating in the development process.
In terms of code quality, Xeridia uses the Sonar tool that integrates into the pipeline as quality controller.
Finally, to be able to perform deployments automatically, a tool must be chosen that can carry out this task. Several options are available, including Puppet, Chef and Ansible. At Xeridia, continuing with the philosophy of providing the highest level of communication, we chose Ansible because it uses a language (DSL) that generates what is known as “executable documentation”. In other words, the documentation itself is actually the code that performs the deployment task, which means we can forget about deployment manuals forever.
THE SECOND WAY: FEEDBACK
At this point, we have already captured the ideas, developed them and promoted them to production. We have followed the First Way of DevOps, accelerating the flow from left to right. Now we need to travel along the Second Way and amplify the feedback from right to left.
We must equip our system with tools to provide information about what is happening in production to all those involved, including members of the IT team, developers, marketing, the business, etc.
In terms of infrastructure, we need to have tools that provide real-time information on how the environment and the application are behaving. Only with this information will we be able to know whether a deployment we just completed is behaving as expected or not.
At Xeridia, we have chosen New Relic as our monitoring tool (APM) because it provides real-time visibility of all the platform’s components:
- Applications and their internal behaviour down to the code level through New Relic APM.
- Associated services such as databases, queues, Big Data platforms, etc. through New Relic Plugins.
- Server behaviour through New Relic Infrastructure.
- Application behaviour in customers’ web browsers through New Relic Browser.
- Mobile application behaviour through New Relic Mobile.
- And all this with a powerful alert engine for the early detection of problems (New Relic Alerts), another for displaying and using all the information collected (New Relic Insight) and multiple integrations with ticketing platforms, Chat(Ops), etc., which guarantee effective monitoring of the behaviour of the environments.
In short, it is a key platform for operations to be able to work proactively in a DevOps context.
But let’s not forget the users. It is particularly important to receive feedback from users as soon as possible and to immediately integrate that feedback into development in order to be able to respond to their requests as quickly as possible.
To do this, we need to look for a modern and flexible support tool that integrates with our ecosystem. In this regard, our use of Jira Service Management provides major advantages, as it integrates directly with Jira and Confluence without the need for additional work.
In certain situations, and with the aim of further facilitating communication with the customer, it could be integrated with HipChat or Stride to provide live support to the user. The advantage of using HipChat is that it integrates with all the tools mentioned above, which makes it possible to implement a ChatOps strategy.
CULTURE OF CONTINUAL EXPERIMENTATION AND LEARNING
Finally, we need to keep in mind that errors do occur and we need to be prepared to learn from them. Blameless post-mortems must be carried out for every project or problem in production, recording what happened and learning from those incidents. To this end, Confluence allows us, using its templates, to document these post-mortem meetings properly, along with the teams’ retrospective assessments, and make them available to everyone. This promotes a culture of transparency, devoid of fear and open to experimentation, which is essential for the continuous improvement of the system. Furthermore, to track problems that occur in the system, Jira Service Management can be used with its problem management workflow.
Together, these tools together help to develop a culture of collaboration and trust between development and operations that makes it possible for the DevOps approach to thrive.
In this post, we have reviewed some of the tools that can be used to implement DevOps. This review is not comprehensive and many other tools can be used in combination, always considering their need to follow the three ways: accelerate the flow, amplify feedback and create a culture of continual experimentation and learning.
The tools are important. However, implementing DevOps is not achieved by using a set of tools alone. The human aspect is also crucial for implementing DevOps successfully in an organisation and is sometimes the most complex part to manage.
In the upcoming posts, therefore, we will also analyse other key aspects for successfully implementing of DevOps.
Emiliano Sutil is a Project Manager at Xeridia
