Dots on a screen.

Eating the Elephant: Choosing a Modular approach to software development

Cover Image for Eating the Elephant: Choosing a Modular approach to software development

Eating the Elephant

There is an old Christmas cracker joke:

Q: How do you eat an elephant?

A: One bite at a time

Software projects can be like eating an elephant.

When tackling any large enterprise it is wise to carve it into individual domains, discrete concerns which may be handled in relative isolation: data, authentication, API, UI. These domains may themselves be broken down into further discrete modules.

Decoupling – isolating the module from other modules and state which do not share their concerns – also helps create robust, fault-tolerant, code.

The advantages of a modular, decoupled approach

There are numerous considerations to take into account at the outset of a project, amongst the most crucial are:

  • Delivery
  • Maintainability
  • Extensibility
  • Fault tolerance

All of these considerations may be addressed by a modular, decoupled approach.

Modules:

  • Do one thing well
  • Are able to be worked on in isolation, allow distributed collaboration and problem solving
  • Are easily refactored in isolation
  • Are more easily maintained an repurposed
  • Have well defined APIs allowing them to be swapped for another module that fulfils the contract of the API
  • May be decoupled from the app as a whole, aiding fault tolerance
  • Allow features to be introduced in a less invasive way than with code monoliths
  • Allow feature flagging and A/B testing
  • Planning modularity – the role of the Principal Engineer
  • All software has an architecture, just some of it is accidental (Dave Farley).

Any joint act of creation needs clear direction from someone with the necessary expertise and domain knowledge, a film director and orchestra conductor. In software engineering this is the role of a Principal Engineer.

It can be tempting to throw a bunch of developers at a project and hope for the best and believe the more developers the quicker the delivery. In practice, regardless of the skill levels and experience of the developers, they will need direction. This should be at a high level – for example: code architecture: where do state, services, Context, styling, utilities, components, pages etc belong Are tests and css kept with components or in a separate test and styles directories

Direction is not micromanagement and developers should be allowed to exercise their expertise and be trusted to make their own decisions at a micro level. At a macro level the applications should follow an opinionated structure.

This direction may change during the course of the endeavour but it should be directed change.

One of the most effective ways to ensure efficient delivery, maintainability, extensibility and fault tolerance, as well as to be able to roll with the punches when there are changes in priority during the life of the project, is to carve up that elephant. Breaking a task into its constituent parts also allows more accurate time and resource and cost forecasting. Separate concerns, use opinionated structures Ideally a code base should look like it has a single author. This does not happen by accident. At the individual file level conventions may be embedded by using linting to enforce code style. At the application level this may be enforced by adopting ‘opinionated’ structures – ones not open to negotiation or interpretation but a set of rules to be adhered to. It is the role of the Principal Engineers to stipulate these opinions for their individual domains.

When planning an application, identify the constituent parts and separate concerns from the outset. On the front end, services, state-management, business logic and UI should all be separate concerns. Keep them in separate directories. At a data/API level employ Domain Driven Design (DDD) principals.

Opinions vary on the best file front-end structure but something like:

src -

  • Components – individual building block components and larger components composed of the smaller building blocks
  • Pages – full pages, composed of components
  • Context – Providers for global context
  • Services – calls to the API
  • Hooks – various state, context and effect hooks

An opinionated structure speeds the onboarding of new developers. When there is a place for everything and everything is in its place – services, components, business logic, utilities, data-transformers, tests – it becomes simpler to track down where existing code lives and new code belongs. There is no real linting for structure, enforcement of application structure is down to code review, clear guidelines for developers and the stewardship or the Principal Engineer.

We are writing tomorrow’s legacy code.

In Japanese cities, where tall buildings are densely packed, it isn’t practical to demolish old buildings with explosives. One solution is to build self collapsing buildings. The first to use this method was The Grand Prince Hotel Akasaka, built in the 1980s and demolished in 2013, gracefully shrinking to ground level, over a period of months, without disturbing the surrounding buildings or populace. https://www.businessinsider.com/tokyo-hotel-shrinks-in-amazingly-efficient-style-of-urban-demolition-2013-2?r=US&IR=T . The builders and architects understood the need for an exit strategy when their creations become obsolete and that individual buildings are expendable and replaceable in the larger context of a town or city.

Most software projects are built as if they, and their constituent parts, will last in perpetuity. In reality software doesn't wear out – it tends to collapse under the weight of its own tech debt and in hard-coupled systems one collapsing element can pull the whole app down with it. Eventually the parts become so obsolete or so overengineered, complex and undocumented that more time ends up being expended on reverse engineering as well as the churn of bug fixing only to introduce further bugs.

Teams servicing tech debt tend to have low morale and high attrition rates as well as long onboarding times as new developers learn the quirks, pitfalls and “where the bodies are buried” in the tangle of code.

So let's build our code like a Japanese city – modular and self contained, with an exit strategy for obsolete code allowing it to be removed/replaced with minimum disruption.

The UI is a Mayfly

The average life of a website, according to research by Orbit Media is 2 years 7 months https://www.orbitmedia.com/blog/website-lifespan-and-you/

With mobile apps the lifespan is even shorter. Axway writes in a 2012 blog post: “...from our own research across hundreds of enterprises, it appears to be a mere 14 months before an enterprise mobile app gets rewritten, and during that time at least 3-4 updates of that app will be produced.”

https://blog.axway.com/learning-center/software-development/api-development/nothing-is-certain-except-death-taxes-and-a-short-mobile-app-lifespan-2

This does not mean the entire site/application is rewritten from scratch (at least in most cases). Apps and websites are layered affairs typically comprising, at the most basic level:

  • Frontend (UX/UI)
  • API(s)
  • Data/databases
  • Hardware infrastructure

Broadly speaking the closer to the front-end the shorter the shelf life for the code, the front-end being the mayfly with its average 14-31 month lifespan. Coupling any of the layers to one another makes future proofing more of a challenge.

Software lifespans

We fill data-lakes with petabytes of data, some of which may live in perpetuity. It is more difficult to get accurate information on the average lifespan of APIs and infrastructure but we know there are RESTful APIs stretching back to the dawn of REST. Following Roy Fielding's dissertation on the proposed Representational State Transfer (REST) architectural style for web services in 2000 The first modern API to use it was the Salesforce API, in February, 2000. This was followed later the same year by eBay and Amazon. These API’s, in an evolved form, are still going strong as are a myriad other APIs which have followed in their wake.

Keeping to the animal theme, in a non-scientific metaphor:

alt text

Hardware infrastructure may have been the longest lived, pre-cloud, but cloud migration to Platform As A Service/Infrastructure as A Service (PAAS/IAAS) has shortened the lifecycle of much current hardware infrastructure. The cloud may yet be another immortal jellyfish, self regenerating when it becomes obsolete.

So, where possible, push business logic to APIs and data worth preserving to data-lakes and warehouses.

When building user interfaces consider reusability and decoupling:

Reusability: Write everything like a library keeping code as generic and reusable as possible Consider writing common library code as NPM packages to allow re-use across projects Use vanilla JS/TypeScript, wherever possible to keep code framework agnostic Keep branding CSS separate from layout CSS. Layout CSS – width/height, grid, margins etc – may be inlined via CSS modules or similar. Branding – colour, fonts, logos etc – are best kept in a reusable format, shareable between projects, or installed via NPM Think in terms of building reusable UI libraries rather than application specific components or use existing UI libraries like Material UI or Bootstrap as a starting point for your own iterations Consider Atomic Design (see https://atomicdesign.bradfrost.com/table-of-contents/) where a granular approach to UI elements allows composition of more complex elements from simple building blocks.

Decoupling: Avoid ‘prop-drilling’ – hard coupling components in frameworks like React, Svelte and Solid by passing shared props from parent to nested child component Keep state management separate from display by using Context APIs or dedicated state-management libraries like Redux or ReactQuery If using Context avoid heavily nested contexts – hard coupling state can be as deleterious as hard coupling UI

The additional advantage of using state management libraries like Redux, MobX, and Akita is they are framework agnostic, totally decoupling state management from the UI and allowing the app to be reskinned, reusing the state layers, even allowing the framework to be changed.

Carefully considering an application’s constituent parts prior to building, subdividing them in a granular way, and taking into account how they may be reused or retired, in a non-invasive manner, helps to reduce the elephant to bite-sized chunks.