Architecting for Change - Part III: Sources of Complexity

Ashraf Mageed

01 Sep 2025 • 5 min read

In my previous post, I examined how complexity and cognitive load can hinder delivery, compromise security, and make systems more difficult to evolve and maintain. To tackle complexity effectively, we must first understand its nature. In this article, I’ll examine where complexity comes from, the different forms it takes, and how some of it can be reduced or avoided entirely.

Types of Complexity

Every software system must navigate complexity, but not all complexity is created equal. By categorising its sources, we can develop more effective strategies to manage, reduce, or eliminate it.

Essential Complexity

Essential complexity is the irreducible complexity that comes directly from the problem domain itself. It represents the complexity that cannot be designed away because it is intrinsic to the nature of what you’re building.

For example, a financial trading platform must comply with strict regulatory frameworks, encode intricate settlement rules, and support nuanced risk calculations. These complexities are dictated by law, market behaviour, and business processes, not by software design choices.

Fred Brooks, in No Silver Bullet, famously described essential complexity as the “hard essence of software” that remains even after all accidental complexity is removed.

Essential complexity can come from:

Domain rules and business logic: Tax regulations, healthcare compliance, airline scheduling constraints.
Inherent uncertainty: Systems that model human behaviour, market volatility, or probabilistic outcomes.
Scale and scope of the problem: A global e-commerce platform must encode multi-currency, multi-language, and multi-jurisdiction rules.
Organisational processes: Business workflows and decision-making structures that reflect real-world operations.

The more demanding the domain, the more exceptions, edge cases, and specialised knowledge your team must embed in software. If unmanaged, essential complexity can overwhelm teams, slow delivery, and increase cognitive load.

You cannot eliminate essential complexity, but you can manage it by:

Structuring systems around clear, loosely coupled, highly cohesive, modular and impermeable boundaries.
Using ubiquitous language to align teams and domain experts.
Creating hierarchical abstractions that model real-world concepts cleanly.
Investing in automation, visualisation, and documentation to reduce cognitive burden.
Applying frameworks like Cynefin to distinguish between complex, complicated, and chaotic parts of the domain.
Aligning organisational structure and technical architecture.
Structuring teams to manage cognitive load, increase autonomy whilst maintaining alignment.

Understanding essential complexity allows teams to focus their energy: instead of chasing unnecessary simplifications, they can embrace domain richness and channel effort into making complexity navigable, not invisible. This series will delve into managing essential complexity in later posts.

Accidental Complexity

Accidental complexity refers to the complexity that is not inherent to the problem. It stems from our solutions, including software design, organisational decisions, tools, and operational practices. Poor design choices, technical debt, and misaligned team structures often make systems far more complicated than necessary.

This complexity often creeps in gradually. Teams add frameworks “just in case” or leave quick fixes in place for the long term. Over time, these choices accumulate into tangled systems that are harder to maintain and understand than the underlying problem ever warranted.

There are a few causes of accidental complexity:

1. Suboptimal Design Decisions (Technical and Organisational)

These are choices—whether at the code, architecture, or team structure level—that make the system harder to understand, maintain, or evolve. They may seem reasonable at the time, but often carry long-term costs.

Technical: Choosing the wrong abstraction, over-engineering, premature optimisation, arbitrary boundaries, or using inappropriate patterns for the problem (e.g. excessive layering or microservices without a need).
Organisational: Aligning system boundaries with team politics rather than domain concerns, or distributing responsibilities in a way that fragments ownership and increases coordination overhead.

These decisions often violate principles such as high cohesion, loose coupling, and modularity, resulting in tangled systems that are difficult to change safely.

2. Tools and Technologies

Accidental complexity arises from the understanding and application of tools and technologies that aid in solving the problem at hand.

Examples include:

Idioms and syntax of a programming, scripting or IaC language.
Using a framework and learning its lifecycle model, dependency injection mechanism, performance characteristics, setup, etc.
Managing Kubernetes and grasping its complexity around service discovery, ingress, and resource management.
Adopting a cloud-native service (e.g. Event Grid, Azure Functions, or Cosmos DB) and understanding its best practices, trade-offs, etc.
Complexities stemming from using the wrong tool for the job.
The proliferation of tools and technologies chosen by autonomous teams.
CI/CD, Security, Automated testing.
etc.

3. Ecosystem and Operational Environment

Systems do not run in a vacuum. They exist in an ecosystem of infrastructure, platforms, policies, and governance. Lack of awareness or deep knowledge about this environment is a significant source of accidental complexity.

This includes:

Deployment complexity: A lack of understanding about aspects such as how APIs need to be onboarded to API gateways, like Azure APIM, or how services are exposed via private endpoints, can lead to last-minute operational chaos.
Security and identity: Implementing authentication and authorisation without understanding identity federation, token flows, or the implications of RBAC in cloud environments.
Networking assumptions: Designing components as if they run on a flat network, only to find out later that service-to-service communication requires VNet peering, DNS resolution, or load balancing policies.
etc.

When engineers lack contextual knowledge of the ecosystem, the resulting system is brittle, complicated to deploy, and fraught with implicit contracts.

4. Drift

Drift occurs gradually as the system evolves and reality shifts away from its original intent or design. It often shows up in the following ways:

Technical debt: Shortcuts taken under pressure accumulate and become harder to reverse.
Semantic drift: A bounded context or service no longer matches the business concept it was designed for—either because the business changed or because understanding was incomplete to begin with.
Technical Misalignment: Architectural decisions that once made sense no longer align with current needs, yet the system remains bound to them.
Organisational Misalignment: Change is inevitable, and sometimes that means the way teams are organised needs to adapt. Reteaming is often ignored as organisational structure and architecture are viewed as static.

While essential complexity cannot be eliminated, accidental complexity can and should be avoided or reduced. Strategies include:

Intentional architecture design: The aforementioned principles for managing essential complexity are also applicable here.
Governance and guardrails: Provide platform services and golden paths to reduce tool sprawl, inconsistent practices, and ensure complexity is avoided.
Team alignment: Structure teams and technical architecture around business capabilities in value streams to minimise coordination overhead and simplify ownership.
Regular refactoring and review: Continuously pay down technical debt before it accumulates and ensure drift is managed early.
Simplicity as a principle: Choose the simplest tool, framework, or design that meets the need—avoid complexity for its own sake.

Conclusion

Complexity is inevitable, but how we respond determines whether it becomes a competitive advantage or a liability. Essential complexity reflects the difficulty of the domain; accidental complexity reflects the mistakes and inefficiencies we add. By understanding these sources, teams can focus their efforts: managing what can’t be avoided and reducing what never should have existed in the first place.

In the next post, I’ll explore strategies for systematically avoiding and reducing accidental complexity.