Bad code scales but bad abstraction does not

Bad code that is properly encapsulated can be contained and managed.

Bad abstraction will bleed into everything and contaminate everything it touches.

The simple concept of Coupling and Cohesion is one of the most important and surprisingly often overlooked concepts in software engineering. Low coupling and high cohesion should be simple, but it is not always the case. It is one of those things that either work seamlessly or lead to hours spent refactoring something that should have been easy.

By properly isolating a system, package, or any conceptual separation with clear boundaries, you can lower the blast radius.

Premature optimization, you say? The popular Go development idiom of "starting out with a main file and taking it from there" does not oppose the idea of proper abstractions. It is not an argument for skipping proper abstractions, and it may not necessarily involve more work. When you find yourself delving into six microservices to refactor a domain object, the frustration will be real.

In a recent experience of refactoring a database integration in a Python project consisting of multiple smaller projects in a mono repo, the choice of using Snowflake as an OLTP database was not ideal. The system's latency and throughput struggled to meet performance requirements, with some requests taking seconds in the double digits. Despite Snowflake not being the most performant choice for OLTP, the organization prioritized a speedy time to market (TTM), making it a justifiable decision given the circumstances. Ultimately, the system delivered value, which was the primary goal.

Upon examining the database integration, one would expect an abstraction between the database and the application, such as a repository to decouple Snowflake-specific code from the application. Additionally, a shared package for applications with similar basic CRUD interaction patterns would have been beneficial. However, this was not the case.

Instead, duplicated code for database interaction was found in all sub-projects, with unique implementations scattered throughout. Most of the code consisted of hard-coded SQL queries, except for one attempt at a data access layer. While the temptation to start afresh was strong, the absence of testing made it a risky endeavor akin to performing a heart transplantation on a live patient. Testing has always proven to be the most effective tool for introducing proper abstractions to a system with minimal effort.

If you cannot test your code, chances are you have overlooked your abstractions.