36
Supercharge ⚡️ your DynamoDB single-table design pattern with TypeDORM
The TypeDORM is an Object Relational Mapper library running in Node,js and written in Typescript. TypeDORM is a feature-rich DynamoDB ORM to help you write high-quality, loosely coupled, scalable, and maintainable applications with highly relational data in the most productive way.
In this article, we will be looking at different challenges that may arise while developing with DynamoDB using a single-table-design pattern and how TypeDORM can help us avoid them.
We will not be covering why and who should use DynamoDB single-table design as Alex DeBrie has done an excellent job at explaining it here.
Once you understand how single-table design works with DynamoDB, you would want to try it out with a real-life example. There is already a very popular article by Forrest Brazeal that can be found here, which you should first look at before continuing with this article.
Once you are familiar with the single-table design pattern and have also looked at how it can be used to model highly relational data in DynamoDB, let's talk about why it isn't a very viable way to work with data and what can we do to make it better.
When first introduced to single-table design pattern, there are some common questions and concerns developers start to ponder:
- I can't use a single-table design, I need operation time "types safety".
- All the attributes and their names just don't make sense anymore, how do I reliably interact with them?
- As I keep adding more indexes, how do I make sure that all the attributes created for modeling don't end up returning to clients?
- If I choose to keep the original attribute value and its index-referenced attribute separate, how do I make sure they are always in sync?
- How do I keep the key schema up to date with a matching pattern for each entity that I add?
- How do I confidently interact with all entities without having to remember to provide correct condition expressions for common operations (i.e for unique attributes, making sure to always include condition expressions with attribute_not_exists)?
- With a single table design, I can't use data mapper libraries.
- The resulting code is hard to read as there are too many attributes/symbols representing different meanings.
- My colleges may know how to work with the DynamoDB but this particular way is very unfamiliar.
- I want to use the single-table design but I don't want to manage each entity schemas in a separate spreadsheet.
There might be many more questions but most of them are pointing to one concern in particular:
"Do I really want to trade off development ease, developer experience, code quality over this new way of working with DynamoDB?"
The answer is: you don't have to. There is a better option.
Let me introduce you to TypeDORM: Strongly typed ORM for DynamoDB - Built for the single-table-design pattern. Let's first have a quick look at what it is and how it can help us to solve all the problems stated earlier.
Later, we will also have a look at how we can leverage TypeDORM to better structure entities (continuing from the example from this great article).
TypeDORM is an ORM built from the ground up using TypeScript to provide an easy entry point when doing complex, highly relational data modeling in DynamoDB. Its syntax might be familiar since the decorator-based syntax is highly influenced by TypeORM - one of the very popular ORM for working with SQL databases.
Let's take a quick look at some of the many features that TypeDORM provides:
- Single table design first-class support
- DataMapper development pattern
- Full type safety enabled using advanced typescript generics
- Declarative relational schema
- Multiple connections support
- Operation based managers per connection, includes Entity manager, Transaction manager, Batch manager, and Scan Manager
- Code follows all possible best practices and provides first-class support for some of the very popular patterns used in the single-table-design pattern
- Additional higher-level features like, such as the ability to declare non-key attribute as unique, automatic retries
- Auto-Generated values for attributes
- Auto-update attributes and its referenced indexes on UPDATE operations
- Dynamic & Static default values for attributes
- Complex update, key condition, and condition expression all made easy to work with
- Powerful expression builder to auto-generate expressions from typesafe input
Let's first install all the modules that we will need.
npm install @typedorm/common @typedorm/core --save
We will also need to install reflect-metadata shim that TypeDORM relies on and configure it so that metadata's are correctly detected by TypeDORM
npm install reflect-metadata --save
If you are using TypeDORM with TypeScript, make sure you also have the below options enabled in your tsconfig.json
"emitDecoratorMetadata": true,
"experimentalDecorators": true,Once this is done we need aws-sdk which TypeDORM uses to communicate with the DynamoDB table.
npm install aws-sdk --save
We are all set now, next we will need to configure the connection. If you have ever worked with TypeORM, this setup will look very familiar.
In the next section, we will have a look at the relational data set that we will be working with. You can always refer back to Forrest Brazeal's article any time you feel lost.
Here are some access patterns that Brazeal has listed:
- Get employee by employee ID
- Get direct reports for an employee
- Get discontinued products
- List all orders of a given product
- Get the most recent 25 orders
- Get shippers by company name
- Get customers by contact name
- List all products included in an order
- Get suppliers by country and region
These are all the entities we will need to create with a given signature to fulfill our access patterns. Here is a slightly modified version of the Entity chart.
The process of selecting what goes under PK and SK remains the same as described here.
If you were following this closely, you will notice few differences between the above entity chart and one presented by Brazeal, Which are:
- Each attribute is only ever referenced in a single index
- Each entity PK now has a prefix of STATIC entity type - this helps us better identify items where they are all put together in the same table as well as allows us to use incrementing ids
Here is the list of all access patterns and what attributes we will be querying on. In case you didn't follow this article and came up with different access patterns, that's perfectly okay too.
Hereafter building on top of Brazeal's access patterns, this is what it looks like now.
You will still have to follow all the steps required in a single-table design pattern, with one slight change being that all the entities need to have a SQL-like entity structure (of course, you can still add more attributes to entities later).
Understanding how to build an entity schema would be a different article on its own - our main focus is how we can reliably work with the above entities using TypeDORM.
This section will focus on getting connections, entities, tables configured for TypeDORM that can effectively parse data.
Since we are doing a single-table design we really only need a single DynamoDB table with a single Global Secondary Index (GSI). We declare that by creating a new instance of Table:
Now it's time to covert our entity chart into an actual TypeDORM entity, we do this by annotating our model with @Entity decorator, exported by @typedorm/common. Product and Employee entities are the only ones listed here, but you got the overall idea.
Note:TypeDORM exposes various decorators to be able to declaratively define an entity. For those not listed here, check out the TypeDORM API specification for more up-to-date APIs.
Product Entity
Employee Entity
You will need to define all entities in a similar way to the above examples.
Once we have all the entities in, the next thing is to initialize a DynamoDB connection. We do this by creating a connection instance:
If all of your entities are grouped under the same directory, you can also provide a relative path to entities option and TypeDORM will auto load all the entities from a given path.
Now since we have everything in place, we are ready to work with a single-table designed DynamoDB table in a type-safe way.
Once everything is set up, it's will be super easy to perform all kinds of operations like Create, Delete, Put, Update, all in a type-safe way.
For example, let's have a look at what our queries will look like for the first two access patterns:
P.S. You can provide additional options to filter/log queries.
For the most up-to-date list of available options and entity have a look at API Spec.
Even though we have only covered querying items with TypeDORM here, there are many more use-cases where TypDORM proves its worth. Experiment with different TypeDORM APIs and tell us what you think in the comments.
TypeDORM is committed to delivering new features every month until all the major document client APIs are covered, meanwhile if you want to access the feature before the release we also publish a beta version.
To find out more about what new features are on the road map, check out the TypeDORM public roadmap.
If there are any other features you would like to be included, open a new issue here.
Thanks for reading! If you liked this article, hit that clap button below👏 . It means a lot to us and it helps others easily find this article. For more updates follow us on Medium or follow me on Github.
Special thanks to Zak Barbuto for proofreading this article.
36