23
Lessons Learned from Multiple Microservice Transitions
I'm unsure if it’s just the nature of the industry at the moment or I've been lucky, but almost every role I've been in over the last seven years has been going through some sort of microservice transition. I've enjoyed being part of these teams and reasoning about turning our hard to maintain, release and reason about monolith into a collection of smaller services.
I've been through this transition in mature startups, large corporate and small businesses. I'm about to embark on another one with another corporate. So, what have I learned? What has worked well? What hasn't? And what am I still figuring out?
We know from SOLID programming principles that single responsibility entities, such as classes or methods, are a good idea. They should have clear inputs, outputs and minimal/no side effects. This thinking applies to microservices as well. A service, in essence, should have one concern. It should do one thing and it should present a contract for consumers to ask it to do those things. As soon as you're designing a new API or service and you hear yourself say something along the lines of "it does X and Y" then alarm bells should go off and you should immediately ask yourself if X and Y should be separate things.
What are the questions you should ask yourself?
- Are X and Y really two different things? Or just two parts of one thing?
- Will anyone need just X or just Y by itself in the future?
- If X no longer needs Y, how hard is it to remove? (i.e Is it too coupled?)
- Will X need different compute requirements to Y?
- Do we need five Ys to everyone one X? How will they scale independently?
- Does X read and/or write data differently to Y?
- Is language A suitable for both X and Y?
- Will the API to X and Y be the same?
- Does X and Y need different environmental configuration?
If the answers suggest these are sufficiently different and have sufficiently different requirements, then make two services. Or more.
There is also a high likelihood that you're using a language that has packages, such as npm packages, ruby gems, nuget, go packages etc. While you're doing your microservices transition, it’s a good time to apply the same thinking to internal packages too.
These would be private packages used only by your team(s) that provide utility. Once again, these should follow similar rules above to determine how much they should do. There is no package too small but there certainly is something as too big. When a developer imports your package as a dependency, they're making their codebase bigger, so they really only want to import the code they need and no more. Small, plentiful packages do that. The irony of a lot of developers is they often use plenty of small public packages to get their work done, but when it comes time to writing their own, they quickly forget the benefit of small packages. Don't let that be you.
A trap that some teams fall into is making one place for tiny utils or common methods. While this may seem a logical grouping, the reality is that even small bits of code should be their own thing. Oh you wanted this one method that you don't want to replicate? Here have 50 megabytes of other stuff you didn't want. Yeah. Not a great experience.
Reading note: From here on out, I'll refer to microservices, handlers, workers and packages as 'components'.
A common trap is that people may try to model components based on what the customer sees, using product or customer domains. However, there are plenty of internal domains that also need to be modelled. For example, a pet website might have a grooming booking API and might have an online store API, but internally we also have a loyalty point API - as both of these public APIS need loyalty support but you don't want to repeat yourself implementing loyalty points in both systems. Implementing those as an upstream domain makes a lot of sense.
In addition to these internal domains, there are also technology domains. Considering the same petstore, an internal technology domain might facade a complex database, or facade an external service, such as salesforce. These domains are critical to model as component(s) as they once again reduce duplication, and increase ease of future development.
Generally when writing code we like to observe DRY (don't repeat yourself), WET (write everything twice) or YAGNI (you ain't gonna need it). A pragmatic mix of these generally serve us well. Making code more generic later when extracting common code from multiple places in the same codebase can often be straightforward.
However, for a transition to microservices, I'd (perhaps counterintuitively) recommend something quite different. As soon as you think that a portion of the component you're working on could be made reusable by other component (a Y is emerging from your X), it's a good time to make another component.
A trap developers fall into, alongside not breaking out a new component is to confuse where concerns should go. A component should never have to reason about who is using it, and the trap is developers end up putting some logic in these generic components actually belong in the consumer of the component. An easy example is to think - does Apple include code in iOS specifically for your app? How about Google with Chrome having code just for your website? No. So pretend you're those companies, building it and you may never see who consumes it (even if you end up being the author of both the component and the first consumer).
So why do this over engineering? Simply put: It moves you faster towards transition completion. Firstly, you're most likely the team member who is the most up to speed on the current task and therefore you can write the seperate component, Y, the quickest. Secondly, Once written, component Y saves one or more other developers time to utilise and build their next component. Now, if you wait until Y is needed in two or more places, the chances for an opportunity from product management to go back and refactor X1 and X2 to use Y is incredibly low. Even if you were allowed, the person doing the refactoring would be need to be brought up to speed on multiple codebases, instead of just one if it was already built and they were consuming it.
An important point for this: Developers overestimate the cost of building it now and underestimate the time it'll save in the future.
So you're doing your task, you've identified some components that can be built and you've built them. How do you not become the person who is constantly asked questions about how to use it, contribute to it and debug it? You improve the developer experience.Developer experience is a topic I've opined on for a long time but with a microservices transition, it's more important than ever and it comes down to a few key steps.
- A great README
- A description of the project/component.
- Local environment variables needed and where to get it (eg a team mate or certain places in various tools, such as cloud or other dashboard)
- How to install dependencies
- How to install anything the project may need (databases, docker services etc)
- Links to read more about major dependencies (eg frameworks, services etc)
- How to run the project
- How to debug the project
- How to run the tests for the project
- Any common traps (but try to solve these in another way of course)
- Examples of usage
- Use Semantic versioning for packages, and versioning for APIs
- A CHANGELOG.md
- List new features, fixes.
- Justify your semver change
- The more info the better-
- Any other info about the project and its implementation should be inside the repo.
A note on Wikis - Wikis are commonly used but in my experience wikis work better across two or more projects. However, if wiki documentation (which tends not to be updated) can be replaced by in-repo documentation, it should be. Like any system or process you're building, try to optimise for chance of sticking to it. Easy to maintain documentation is the best documentation.
Ideally, your project should be very accessible - as a goal, an intern should have minimal trouble setting up. Set them up for success with your great developer experience.Once again, people overestimate the cost of adding good DX to their projects and underestimate the time it'll save others in the future.
One technique for easing the burden for this is using Github's template repository feature. You can create multiple templates. Consider setting up one for services, one for packages and any other common project type, using the tools your team have chosen.
I must recall an exceptional template a colleague set up at company I used to work at. It was for TypeScript projects, with typescript, jest, Azure CI yaml file, Dockerfile and a templated README all in place. The experience of using this delightful. To get started on a new project, it really was as simple as creating a new repo, based on the template, clone it, find and replace 'template-repo' with the component name and then you were good to go. The template also had real code that had real tests, so it’s easy to see how testing should work. If you're not on GitHub, I'm sure there’s a similar solution you can use with your Git provider.
A transition to microservices, as tempting as it may be, is not something you form a team around to do for a year or ten. You should instead increment towards microservices. You can do this one project at a time. This allows you to learn as group, experimenting a bit with separating your concerns and generally bring everyone along on the journey. This also means you remain agile, ready to adapt and evolve as your business requirements may change.
Finally, I want to finish off with talking about naming things. Depending on how big your software is, you may end up with 10s if not hundreds of components. You should therefore chose a naming scheme that scales. The naming should be brutally obvious.
Here are some examples of some DOs:
-
email-sending-service(Very obvious) -
loyalty-service(Obvious that it handles all things loyalty) -
@company/something(for npm packages) -
X-service(Services respond to requests via HTTP)- -
Y-worker(Workers or Handlers respond to messages from streams and queues)
Examples of DON’T:
- The Mailman (What? Email maybe?)
- Stamp My Card (Haha I see, loyalty)
- Twenty Questions (Is this for querying stuff? From which database? Is there really a rate limit of 20 or is that part of the humour of the name?)
- Locke And Key (Auth*? ugh. That netflix show from 2020 will be with us for a while then) uuid (is this our package or the one in node or another one?)
As you can see, creativity and humor has no place when it comes to naming at scale. By all means, have a fun code name or team name, keep the humour at work, but naming brutally obviously will serve you very well.
So those are just some of the lessons learned as I've done microservices over and over again. I actually love the journey. It's always awesome to see people have that "ah ha!" moment as they use their component a second time without difficulty, or smile when someone uses their work without assistance. These lessons and best practices I've observed have a try it and see nature but I tend to see the same results. Developer happiness increases. Craftsmanship increases. Finally, Output and efficiencies increase. If you're starting or going through your own microservices journey, I wish you the best of luck. I hope this can help you a little on your way.
What did you learn during your journey? What have I missed? Let me know in the comments :)
23