Software Design: Defining Complexity When Discussing Technical Approaches

Can teams build a shared understanding of complexity, so technical discussions can be productive and satisfying?

Introduction

It’s typical when starting work on larger features, a new system architecture, or a substantial technical initiative, to have some kind of discussion in the team to validate and share knowledge about the intended approach.

  1. Unstructured exploration of complexity with mixed conclusions
  2. Impasse and frustration, with no common understanding of the complexity

1. How difficult is it to describe this approach to another engineer?

Even extremely talented engineers can fail to provide context, establish the key abstractions being affected by a change or highlight the strengths and weaknesses of an approach. If you find that a conversation has jumped immediately to code, and you’re watching an engineer highlight functions and explain their purpose, then it may be an indication that a step back is needed.

2. Are we using the right abstractions to solve this problem?

Often, it’s necessary to reflect on the complexity built over time in a system. This can be the result of a number of tactical decisions made, that are beginning to cause drag on the system. If the symptoms of this are becoming clearer:

  • Obviousness — Multiple ‘gotchas’ involved in making a desired change that must be navigated cause high cognitive overhead.
  • Overexposure — Unrelated concerns must be taken into account to make a desired change.
  • Special / General Mixtures — Behavior that is specific to one set of user-requirements pollutes other general purpose capabilities.
  • Depth — How much complexity does this abstraction hide?
  • Purpose — Does this abstraction simplify a cohesive problem? Multiple problems?

3. How difficult is it to change this approach ?

This question is aimed at facilitating a discussion on risk, along the lines of continuous experimentation principles. It is common in software engineering teams for there to be tension between:

  1. Exploring new approaches — Typically driven by the ‘engineer’ persona in a discussion, a curious mindset always seeks to find better ways to solve a problem, potentially discarding previous investments.

4. What dependencies does this create for the team ?

Build systems, code libraries, frameworks, languages, cloud-provided services, these all create dependencies and constraints for the team. Spending time deliberately discussing these dependencies can help to identify pitfalls that may manifest over longer timelines in a software system. When discussing dependencies, consider:

  • Licensing model for new dependencies, engineering support, community and velocity. Will any costs paid for a dependency scale as the system grows ? Is there enough support in the community and enough development happening on a core library ?
  • How opinionated is a framework ? Frameworks that are overly prescriptive may not be suitable for systems which have monolithic properties. Discuss how the scope of a problem is likely to change over time, and whether the framework facilitates the intended growth.
  • Negative freedom — The freedom from things happening to you, e.g. the freedom from having to support additional programming languages (even if others would prefer to use them).

5. How are we affecting the cognitive load of the team?

Supporting autonomy and mastery in a team can be a question of providing well defined, correctly sized responsibilities and boundaries. Try to be conscious of this when making decisions that affect this boundaries and be mindful of overloading the team. It may be practical to rebuild a component in the team, vs a more limited off-the-shelf solution, but does it overburden the team in terms of maintenance and cognitive load ?

  • Too many different ways of achieving the same thing. For example using ECS & EKS at the same time.
  • Proliferation of programming languages. Yes certain languages are better at specific problems, but each language takes time to master. Keep the number of languages in use appropriate to your team size.

Conclusion

Being deliberate in your discussions around complexity, and applying structure to these discussions, can help establish a design ritual that is satisfying, engaging and valuable.

@MikeOkPatel on Twitter. Engineering Director at Genius Sports Media. Views expressed on this blog are mine, not those of my employer or past employers.