26
Why start hyper? (Unintended Technical Debt)
After 20+ years of creating software and building technology teams using different technologies, frameworks, and skillsets, a common pattern emerged. This pattern was not just specific to our teams; it was also commonplace among the software industry.
The pattern? Unintended technical debt
As a software project evolves, productivity decreases while the cost to develop increases. Over time, more developers accomplish less.
As developers, we should be able to design and build software in such a way that teams can stay equally productive on day 451 as on day 1. But over time business rules change, user interfaces shift platforms, the need for custom integrations create unexpected architecture changes, complex data relationships emerge, comprehensive reports create performance bottlenecks, etc.
A successful project can bring a surge of constant change. While these changes are being developed, increased thought and attention have to be applied to the existing system. As more people are placed on the project, discovery boundaries rarely get established, so experiments tend to get tried and abandoned. There is a focus on measurements that tell us what is happening — not if the changes we make improve the system.
Pascal, C, Basic, C++, Java, CSharp, Python, Ruby, Haskell, Erlang/Elixir, JavaScript, Go, Lisp/Clojure, Typescript, and Rust languages all promised to provide a more productive experience with better features than other languages.
There will always be the next Language X. Static types vs dynamic types, compile-time vs runtime, garbage collection, multi-threaded, low-level vs high-level, virtual machine runtimes, etc. Every language has features and foot guns, the reality the language choice does not move the needle that much in terms of productivity.
What is a foot gun? I foot gun is a common termed used around language defects that can appear to be a great solution but leads the developer into trouble. (in other words, shoots you in the foot)
You need a team with a solid understanding of the language, and there are always special needs that one language may do better. Using a particular language does not avoid unintended technical debt.
In the last 20 years, there have been several frameworks all with the same promise. Don’t re-invent the wheel, use Framework X and you will get your project up and running fast and stay productive over time. The speed to get a software project started improved with every new iteration of a given framework, where frameworks start to break down is the handling of specific cases that may be outside of the fairway of a given framework. Integrations with specialized libraries to handle specific cases like maps, and graphs. Over time the dependency on the framework increases and if your project outgrows the boundaries of the framework productivity can suffer because you begin to deal with unintended technical debt.
Frameworks don’t move the needle in terms of eliminating unintended technical debt, they help teams stay on the same page, but usually guide you to a popular paradigm that tightly couples your business rules to the framework, making it impossible to grow out of the framework without a re-write.
The problem is not with the technology your team is using, the language, or the framework, it is the process, right. The way you manage your software development life cycle, balancing features and chores and defining the scope and timeline. If we had project managers, or product managers and scrum masters, divided our team into two teams a new features team and a maintenance team, or a frontend team and a backend team, or a team focused on each domain. We could be more productive.
The bottom line is how we manage our communication is the key to staying productive.
While having a good process helps, it is not the silver bullet that some pretend to sell. A good process is great for keeping everyone on the same page, but regardless of process, if you have a mountain of unintended technical debt that continues to grow every day, you will continue to have challenges staying productive no matter the process.
When a small team starts a project, they leverage the best languages and the best frameworks for the job. Over time, they start to encounter slower productivity, so they increase the team size, the need for communication increases, and the need for everyone to stay on the same page. The workload starts to specialize so multiple teams are created and more communication is required, on and on, until everyone feels SAFe 🙂.
Like many developers, I got into distributed computing, it would be great to just dispatch a request on an information bus, somewhere down the line a specialized process would be waiting to fulfill that specific purpose and report back on the bus.
Or the microservice, small little subsystems that are singularly focused on performing a specific function. With its own persistent storage and its own development lifecycle.
Both patterns can solve some pain points, but they come with a cost, if a single team is trying to manage several microservices, it can be taxing on that team to have to manage different deployment pipelines and persistent storage layers. If the services are in different languages or frameworks, you can lose the ability to transfer team members from domain team to domain team.
I don’t think breaking the application into services improves productivity as much as it displaces accountability. Creating solid well define boundaries that allow for truly independent software life cycles is imperative, otherwise, you have implicit coupling between services and teams can be left waiting on other teams to complete their tasks. Focus on design and systems thinking to create solid boundaries between systems is key. Distributed architectures require more communication and collaboration with various teams to keep services in sync, and benefit large organizations that can assign specific teams to specific services.
How do we solve this challenge of building software and keeping technical debt low and productivity high, to keep from having to increase the size of the development team with every 10 features?
All the above ideas and patterns work and help, I think they address symptoms and not the root cause.
I think there are a couple of mindset changes we can make as developers to dramatically improve the productivity of the software development life cycle over time and keep the development cost down.
This is why I founded hyper, a company focused on solving this problem, by providing services and patterns that change a developer’s mindset about how to design software and build software systems. This is not novel or magic, many leaders in our industry have been sharing these patterns in the form of principles, architectures, and paradigms, but the status quo far outweighs these pragmatic concepts.
- Functional Thinking
- General to Specific (Clean Architecture)
- Continuous Delivery
Applying these core concepts, software developers can create high-performing development teams and cost-efficient software maintenance. Each of these concepts requires discipline and commitment to the principles and practices of each concept. So they are not easy things to do, you have to change the way you approach solving problems.
I got the term functional thinking from Eric Norman and his book Grokking Simplicity. The core idea of functional thinking is to apply the functional programming paradigm to any language by simply changing the way you think or approach solutions to software problems. The basic concept is to organize your codebase into three common areas of concern, actions, calculations, and data. These areas of concern help you separate business logic from implementation details. Then work with immutable data structures, in functional programming all data structures are immutable, you can implement immutable data structures by implementing copy on write and defensive copying within your functions. There is much more to this concept, but the gist is that you want to separate the core components of your application into these three categories (actions, calculations, and data).
- Data — facts — functions that return data
- Calculations — or commonly referred to as pure functions, these functions take input and return output, they do not modify anything outside of their local scope
- Actions — everything else, they are components and functions that handle user interaction, or communication with external services. Some may refer to these components are impure functions
The idea of Clean Architecture is to separate your implementation details from your business logic. This concept creates strong boundaries between interfaces, services, and core business rules. I like to think of it as a way to divide general-purpose services and interfaces from specific business logic and rules. This separation creates a stratified design that separates your code into layers with the code that changes the most at the top and the code that changes the least at the bottom. When paired with functional thinking, this separation can provide productive semantics that keeps the cost of changing a business rule as low as possible. A side effect of this pattern is that it provides the ability to swap or exchange service or interface without having to re-write all specific business rules.
Continuous Delivery is the process of automating the software lifecycle delivery process and applying and managing four key lead measures to improve quality and continuously pay down technical debt. Automating the process of delivery pulls several requirements into the development process, automated reliable testing, frictionless quality gates of control, self-monitoring infrastructure, etc. Four measures help provide a guide to the performance level of a development team.
- DF — Delivery Frequency
- MLT — Mean Lead Time for change
- MTTR — Mean Time to Recovery
- CFR — Change failure rate
These are lead metrics that the development team can monitor and track to gauge their performance. The DORA initiative showed using science that test metrics reinforce quality and keep technical debt low by constantly delivering change over time, by leveraging tools like feature flags development teams are empowered to ship code daily without having to wait for the entire feature set to be completed and signed off.
hyper was born to build services and products that guide developers into embracing and leveraging this software development mindset, from the three concepts, Functional Thinking, Clean Architecture, and Continous Delivery.
hyper’s cloud service and open-source meta-framework create a clear border between general services and implementation details, using an API your application can access all of the general services it needs without adding specific business logic to your services layer. The result of this separation is that it removes a burden of maintenance from the development team and makes it much easier to change business logic over time.
hyper is not your typical backend as a service, hyper creates a boundary between general (implementation details) and special (business logic). By creating a boundary between the services and application layer, unintended technical debt is front and center and needs to be addressed.
For more information about hyper’s architecture — https://blog.hyper63.com/hyper-architecture/
At hyper, we are in the process of building API kits, these kits will show developer teams how to leverage the hyper service for multiple use cases. The API kits will also provide a blueprint on how to apply functional thinking, clean architecture, and continuous delivery to API applications.
Thank you for taking the time to read this background piece about why hyper was founded. Our mission is to attack technical debt at the root cause and show teams that building software can be a joy from beginning to end of the software lifecycle. If you want to learn more, check out our blog https://blog.hyper63.com or subscribe to our youtube channel or get in touch at [email protected].
26