Agile promotes quick validation of value hypotheses based on working software

Agile as a term was born in 2001 when the Agile manifesto was published by 17 software development enthusiasts. Most of them became leading figures in the industry and made a significant contribution to it. Their main focus was on “uncovering better ways of developing software by doing it and helping others do it” as it is said in the manifesto. They developed the Agile approach, which became more and more popular over time and even went beyond the software development domain where it was initially born.

Popularity brought a number of misconceptions, so it’s important to go back to the roots if you want to get to the bottom of the matter. The whole Agile concept is based on the sixteen statements which are divided into four values and twelve principles. I'm not going to repeat all of them, but will refer to those which give me enough context to elaborate the question raised in the title of this article.

Being Agile means that you value responding to change more than following a plan. The plan still exists, but the idea is to be responsive to the continuously changing environment (market, customers, users, competitors, etc.). To achieve this, you need to check your ideas, assumptions and hypotheses against reality. Ideally, you should be able to perform such validations more often than your environment changes and in order to be in a safer position you need to do it as often as possible. The third principle of the Agile manifesto says: “Deliver working software frequently, from a couple of weeks to a couple of months, with a preference to the shorter timescale.”

Agile requires a fast reaction. The shorter the cycle time between the decision to develop something and inspection of the expected value in the field, the less time is spent on moving in the wrong direction. However, it’s not a good idea to release a feature for feedback, then receive complaints regarding other features that have been broken by the release. It’s not enough just to be able to deliver fast. The delivery process should be reliable and sustainable. “Agile processes promote sustainable development” and “working software is the primary measure of progress” as two other principles declare. In other words, software should always be in a workable condition.

Integration of code can be a bottleneck and slow down the delivery flow

How do we ensure that software is in a workable condition? We test it thoroughly, right? There could be a whole integration testing stage to validate all the features implemented by different developers working together in the end. Those who have ever experienced this approach know this could last for days, weeks or even months depending on how long developers worked in isolation. Unexpected delays, close deadlines, and increasing number of defects generate demotivating tension. It may be a true “integration hell” which might take too much time to solve.

If the integration process takes more time every delivery cycle, you can validate added value less often. You can be less agile in this case. To mitigate this risk, integration of code should go smoother and faster to avoid this bottleneck. Ideally, we should avoid accumulation of changes developed in isolation. Their integration takes too much time due to accumulated conflicts in code, design or even architecture concerns. The number of conflicts grows progressively along with the time developers work on their features in isolation. It basically means that more rework is required to resolve these conflicts and it takes more time for the whole delivery cycle as a result.

The idea is to minimize the amount of time that developers work in their bubbles not knowing what is happening with the rest of the codebase. If everyone can see changes right after they are applied, it’s possible to avoid most of the potential conflicts and even if they happened, there is a great chance to resolve them early, before more conflicting code is produced.

Instant integration of text in Google Docs

Google Docs or any other live document tool is the ultimate example of how contradictions can be avoided or resolved in a timely manner. Everyone sees modifications done by others in live mode. If several contributors work on the same part of the document — it’s visible right away, so they can agree on the way forward before any more significant changes in the opposite directions are made.

There is no need to merge text from several documents prepared somewhere else locally. There are no exhausting and long comments to resolve contradictions caused by this merge, and no need to re-edit the final version to make structure and style consistent across the whole document. Thus, no excessive time is spent.

It’s also important that the document is prepared in such a live-mode can be ready-to-read at any time. It doesn’t contain any major conflicts, so there is no need to wait until it is merged, reviewed and finalized. It perfectly reflects its current state.

“Instant integration” is a lack of integration at the extreme. Every change is already integrated when it is done. The whole thing is always as close as possible to the ready state while wasted time and effort are minimized.

Continuous Integration of code is an extreme engineering practice

The idea of the “Continuous Integration” as one of the eXtreme Programming (XP) practices is to perform integration of code often (several times a day) so every developer can see and consider the most recent changes performed by others in the mode which is close to real-time (depending on how often developers commit their changes to the mainline). Potential impacts and conflicts are visible early. The rule to commit frequently forces everyone to merge small pieces of code which just can’t contain too many issues. And in case of any problems, they can be easily localized and quickly fixed.

In fact, integration issues must be fixed quickly, within 10 minutes. Everyone commits to the common main branch and uses it as a base for developing their next portion of change. If the mainline contains bugs, future development is more likely to be impacted. Developers may be just blocked or rely on wrong system behavior when implementing new features. That’s why it is vital that a maximum number of issues are found after every integration cycle, for example after every commit to the mainline. Basic strategy for fixing a broken build is to roll back the unfortunate commit and investigate the problem separately. Software that is built from the mainline should always be in a workable condition.

Automate everything you can

How can you ensure that software built from the main branch is in a workable condition? The appropriate tests should be applied to the build. Considering that integration of code should be performed several times a day, it means the same number of builds should be tested within a day.

As we discussed earlier, Agile is about the speed of reaction and minimization of cycle time. Slow steps should be eliminated or replaced by faster ones. Automation is one of the obvious strategies which should be considered here. Manual operations are the first candidates to be automated. Usually, they are performed several times slower than a machine can. They also include the risk of human error, which is not the case for an algorithm.

Build automation is the first concern on the way to Continuous Integration. Software should be easily and quickly built by a simple command. The command can be triggered manually or automatically right after every commit. The ultimate goal is to avoid any repetitive manual activities that are supposed to gobble up time several times a day, thus stretching the process.

Test automation is the second important component to pay attention to. There is not much sense in the rapidly built but buggy software. If we can’t release new updates to customers and users, we can’t validate our hypotheses about the value these updates potentially bring. As a result, we’re missing the opportunity to learn and adapt until the issues are resolved. That’s why the Agile approach promotes sustainable development and refers to workable software as a primary measure of progress.

The intention is to think of every tiny commit as a software product increment which results in a new release candidate. Every release candidate should go through maximum sets and types of tests (e.g. unit tests, integration tests, functional tests, performance tests, etc.) to be sure that any commit doesn't add issues to the software product together with the expected value. Tests should be performed often and fast, and results should be available and visible for everyone as early as possible. That’s why the maximum number of tests should be automated, especially considering the fact that their number permanently increases with time leaving almost no way for manual testing often and quickly.

It is worth mentioning that some tests can’t be automated by their nature or because their automation and support are too expensive. Some tests may take a lot of time even when they are automated. In this case, several stages of build testing are typically organized where the first stage takes no more than 10 minutes and contains the most crucial tests to be performed. But, the intention should be to automate and include as much as possible in the first stage in order to have more opportunities to discover issues sooner rather than later.

“Scheduled” is not “continuous” even if it is done every night

Achieving a high rate of automation is not easy. Organizations and teams try to compensate for it by scheduling builds and tests to run at the end of every day or overnight. Of course, it’s better than less frequent integration periods but it still can’t be called continuous integration.

The whole point of CI is to discover issues as soon as possible. If you postpone this validation to the night, you leave more potential for the issues to be accumulated and to be impacted by each other. It will likely take more time to resolve them. Additionally, having more commits to merge makes finding the root cause of the issues more difficult and time-consuming.

Scheduled integration negatively affects overall cycle time and limits the opportunity to inspect and adapt. New commit is the earliest event to trigger the build and launch its validation. When it is done after every commit the process of integration can be called “continuous”. Continuous validation ideally supports Agile principles.

CI makes more sense for Agile with CD

Many people know Continuous Integration practice only within the “CI/CD” term. Well, it’s really an inseparable pair and there is a good reason for it. There is not much sense in integrating code fast and often if new features are not delivered to users and customers at a comparable pace. Agile is not about just IT. In the end, it is about delivering business value to customers.

Continuous Delivery inherits the approach to Continuous Integration but scales it to the whole delivery cycle. It means that any change in your software product should be ready to be delivered to users and customers at any time. Changes can even be continuously deployed to the production environment when all the corresponding test gates are passed. Features can be released right away or turned off by feature toggles and released on demand when it’s time to do it. Any approach that provides an ability to release at any time works fine here.

CD is such a logical extension to CI in terms of Agile, that they shouldn’t be considered separately. All the above sections of this article talk about CI while always keeping CD in mind.

Continuous Integration Summary

Continuous Integration practice by itself provides a full range of advantages such as smaller portions of changes to validate, easier localization of problems and recovery process, fewer contradictions, minimized rework, as well as less confusion and tension before every release. However, the most powerful benefit of Continuous Integration when paired with Continuous Delivery, is the reduced risk of doing wrong things which doesn’t bring the expected value to your customers. By reducing this risk of moving in the wrong direction, we are getting closer to the state we call Agile.