Agile development is about quickly responding to changes.

The term “Agile” takes its roots from the Agile Manifesto written by seventeen enthusiasts and leaders of the software development industry in 2001. This manifesto for Agile software development consists of only four values and twelve principles, but there is an infinite number of articles that try to explain the ideas behind them, with varying degrees of success. The goal of this article is different, so I would like to focus on what is important in the concepts of Agile from the architecture point of view.

One of the basic preconditions for the Agile approach to software development is that quite often we have to work in complex, rapidly moving environments where:

1. Impacts are not very predictable

2. Changes are inevitable

3. The future is quite uncertain

In these circumstances, it looks logical to avoid overly-detailed long-term plans and designs that should be followed and focus on how to validate assumptions with fast feedback loops and adapt to changes quickly. The ultimate goal is the value that we bring to our customers. Even though this goal does not depend on the approach, it is important to keep it in mind.

Architecture is defined at the beginning and hard to change later.

The term “architecture” came to the software development industry from the construction industry. The most common analogy is building architecture. Some experts like to compare software architecture with city planning, which gives more freedom in terms of implementation because of a bigger scale. Nevertheless, architecture seems to be some structure of elements, relations between them, and their properties that are taken as a base to build something real and usable.

Building architecture is strongly associated with blueprints. In most cases, we assume that the building construction cannot be started until all the calculations are done, and the blueprints are finalized. After the blueprints are finalized, and construction is started, it’s unlikely that we get another building at the end that’s different than the one envisioned. So, architecture for the building is defined before construction begins, and is strictly followed thereafter.

Similarly, software architecture is usually represented with structure diagrams which take the role of blueprints. These diagrams typically show a system’s components and relationships between them. So far, so good! The trap is hidden in the mindset attached to such kinds of “blueprints” which make us think that we should make all the decisions that are hard or costly to change before we can proceed further.

This mindset is well-justified for a building that is going to be erected in known and stable circumstances (location, type of ground, environment) but for many software products, it’s not the case. There are numerous examples of when applications completely changed their shape during development. So, software architecture should provide some freedom in terms of the ability to make such pivots, but before we elaborate on the freedom, let’s talk about the constraints.

Architecture comes from and produces more constraints.

1. Time, money and people

2. Given IT environment, integrations

3. Non-functional requirements

4. Best practices, standards and guidelines

5. Frameworks, libraries and tools

What do they mean in your context?

1. Some constraints can be fixed. For example, we can have a certain budget for a certain team with a certain deadline. If the product must be compliant with legal requirements, this is also a constraint that cannot be avoided.

2. Some constraints are assumed to be fixed, but they are in fact negotiable. They can be a subject to push away when considered carefully. For example, non-functional requirements (such as security, reliability, performance efficiency, maintainability, portability, scalability and usability) are often applied without conscious consideration of the value they bring versus the effort required to support them. Indeed, “99.99% availability” and “99.98% availability” are really different constraints.

3. There is an opposite case when some constraints are not explicitly considered or ignored (intentionally). Sometimes it is justified by the myth that Agile teams don’t need to think about architecture. Let’s just start delivering software and think of the problems as they arrive! Indeed, the future is unknown in complex environments, but we can make decisions based on what is known. If we can gain some knowledge to make a better decision, there is no reason to refuse it.

4. There also might be accidental implicit constraints. For example, if you start to develop a new product with certain people, you can become a prisoner of their knowledge, experience, personal goals, professional paths or wishes. Some of them can just be tech geeks with a tendency toward various IT trends. This can lead to unforced architectural decisions. One of the ways to manage this is to make such constraints explicit and discuss them openly.

The more explicit constraints there are, the better the chances to avoid inappropriate architectural decisions. Since every architectural decision leads to new constraints, they should be considered carefully in Agile software development. The fewer limitations the architecture has, the more freedom the software product has to adapt to the changes.

Agile architecture should support the evolution of software.

The ability to adapt is one of the key properties of Agile architecture. However, the level of this ability may vary from product to product. It can be an art to find the right balance between intention and emergence and hold this balance through all the stages of the system's evolution. Agile is an expensive thing, so every decision should be reasonable based on current circumstances, and the value expected to be gained. “It depends” and “context matters” as the sayings go, but it may be helpful to keep some thinking patterns in mind to increase the chances of Agile architecture becoming your reality:

1. Avoid irreversible architectural decisions. Find a way to make them reversible instead. Some time ago it was quite common to design a data model and agree on it before the development started. The reason is that the database structure is hard to change after. Thus, we get the constraint that is based on many unvalidated assumptions. One of the ways to mitigate this is to introduce additional levels of abstraction which can absorb some changes, but this makes the solution more complex. Another way is to simplify and cheapen the process of such changes. There are tools (e.g. Rails migration) for easier changes in database schemas, which eliminate the constraint and don’t add complexity to the product itself. We can change our mind later without a significant impact.

2. Propose options and preserve them. Since nobody knows the future, especially in complex environments, it’s unlikely possible to know in advance which architectural decision suits best of all. Several options that are validated in parallel give a more clear picture of the pros and cons for each option and leave more space to choose the most appropriate path. The options can be found and validated via investigation spikes, models and prototypes. The closer to real-life conditions the experiments are, the more real knowledge we gain and the better choice we can make at the end.

3. Postpone final decisions until the last responsible moment. Developing and preserving options provides variability, which means a bigger room for action. Postponing the decision provides time for reflection on the empirical data collected and more conscious action as a result.

4. Revise architecture regularly. There should be a cycle to reflect on the previously-made architectural decisions and constraints, challenge them against current circumstances and business goals, define current pain points and enablers for future development, and agree on the plan to evolve your architecture. Do you need to move your product from a monolithic application to a service-oriented approach to support new customers who require only parts of your solution? Does it make sense to move from the self-hosted environment to the cloud to resolve constantly experienced hardware scaling issues? Architecture should be an integral part of the evolving product roadmap.

Agile architecture should support fast feedback loops.

Agile requires a fast reaction. The shorter the cycle time between the decision to develop something and inspection of the expected value in the field, the less time is spent on moving in the wrong direction. The ability to respond quickly is the second key concern of Agile architecture.

There is no magic in getting shorter cycles. It’s just a result of systematic optimization of the delivery flow. Every team or organization has its own ineffective parts of the flow that require individual consideration. However, there are also quite common obstacles that fundamentally impact responsiveness. The good news is that they can be fixed with appropriate architectural decisions.

1. Dependencies. If every time you need to release a simple feature, some updates within other parts of the system are required, there is a potential architectural improvement. Ideally, the system should be decomposed in a way that every component is deployable independently and can be swapped out for another implementation with no impact on other components. The less coupled parts of the system we have to touch to make the change, the faster we can release it. Service-oriented architecture (SOA) and Microservices architecture styles elaborate this concept.

2. Instability. It doesn’t seem like a good idea to release a feature for feedback, then receive lots of complaints regarding other features that have been broken by the release. It’s not enough just to be able to deliver fast. The delivery process should be reliable and sustainable. Continuous Integration (CI), Continuous Delivery (CD), and Continuous Deployment ensure the “always-ready-to-release” state of the whole system. Automated tests, monitoring, and an automated deployment pipeline help to discover regression early and often. However, it’s not going to go well if developers merge their code to the trunk less often than daily or more than ten minutes is required to return the broken build back into a working state. So again, the system should be structured and decomposed in a way that allows developers to merge changes often, quickly isolate and fix issues or revert changes using API versioning.

3. Hardware. Extending infrastructure capacity can be a real bottleneck in the delivery flow. The process of acquiring additional hardware takes time and can easily be several times longer than your regular delivery cycle time. Interestingly, it is not always really required, especially if you experiment with some hypotheses. In addition to losing time, there is a risk of wasting money. Moving to the Cloud can be an option here. It doesn’t necessarily mean the total and ultimate migration. This decision can be made for the most demanding parts of the system.

It is worth mentioning again that Agile architecture is a subject of step-by-step evolution. For instance, if it seems that there is a value to move from a monolithic application to the cloud-based microservices, it should be done by taking the most demanding parts of the system and converting them to the new format. Then, monitor how it’s going before selecting the next portion and so on.

How do you validate your architecture against Agile?

The metrics naturally emerge from the key properties discussed above:

1. Exploration and visualization of the code coupling can provide insights on dependencies that make particular parts of the system hard or risky to change. They probably require some refactoring. Monitoring the dependencies between components or services does a similar job.

2. The average number of merges per day in the trunk, the average amount of time to repair the broken build, the average cycle time to release, and how often changes lead to regression can highlight some architectural issues.

3. Time is required to introduce, configure, scale, upgrade, replace and remove hardware or software including third-party solutions like libraries, frameworks or tools. This can reveal potentially weak links in the chain.

4. How regularly the architecture revision is completed, and the percentage of architectural spikes and experiments compared to new features development can indirectly show the tendency of the architecture to be adaptive.

Being Agile has its costs.

Preserving variability of options, automation on various layers of the delivery pipeline and microservices are examples of expensive practices from both financial and organizational perspectives. It requires direct investments and a high level of organizational discipline so it’s vital to keep the right balance between the cost and the value that agility provides.

Who is responsible?

Responding to changes quickly demands the elimination of bottlenecks in the flow. One kind of bottleneck can be a senior or lead architect who should validate or even approve all the decisions which seem to be architectural. In big enterprises, it can be a group of architects, but the organizational pattern issue is the same — it usually causes delays. Another side effect is that these guys, who have to spend most of their time thinking of “global concerns,” are stuck at this high level which can be too far from the real problems in the field.

The more decisions that are made in a decentralized manner, the faster the flow. It doesn’t mean the ultimate freedom to do whatever you want. However, it’s vital for the decentralized decision-making approach to have clear boundaries and guidelines. Another important ingredient is knowledge sharing and transparency over any changes or updates. It requires a high level of responsibility and discipline, which implies a particular organizational culture with specific personal values. Yes, it’s hard, but as one of the principles of the Agile Manifesto states: “The best architectures, requirements and designs emerge from self-organizing teams.”

How do you start with Agile architecture?

If you are halfway down the path and realize that your architecture doesn’t support Agile, it’s difficult to define any common recommendations. You have to find your own path to evolve. If you are at the very beginning with a strong reason for Agile architecture, then don’t forget to:

1. Define your constraints, make all of them explicit, push out as many as possible.

2. Define basic components/services. Domain-driven design (DDD) can be useful here.

3. Validate the components/services against dependencies.

4. Agree on architectural guidelines, roadmap, and the revision cadence.

5. Self-organize into teams if you are at scale. One service — one team.

Best of luck as you begin to explore Agile architecture. If you have further questions outside the scope of this post, feel free to contact us at RIIS. We are always happy and ready to help.