Efficient DynamoDB in Kotlin with Tempest

Efficient DynamoDB
DynamoDB applications perform best (and cost the least to operate!) when data is organized for locality:
  • Multiple types per table: The application can store different entity types in a single table. DynamoDB schemas are flexible.
  • Related entities are stored together: Entities that are accessed together should be stored together. This makes it possible to answer common queries in as few requests as possible, ideally one.
  • Example
    Let's build a music library with the following features:
  • Fetching multiple albums, each of which contains multiple tracks.
  • Fetching individual tracks.
  • We express it like this in code:
    interface MusicLibrary {
      fun getAlbum(key: AlbumKey): Album
      fun getTrack(key: TrackKey): Track
    }
    
    data class Album(
      val album_title: String,
      val album_artist: String,
      val release_date: String,
      val genre: String,
      val tracks: List<Track>
    )
    
    data class Track(
      val track_title: String,
      val run_length: String
    )
    We optimize for this access pattern by putting albums and tracks in the same table:
    Primary Key Attributes
    partition_key sort_key
    ALBUM_1 INFO album_title album_artiest release_date genre
    The Dark Side of the Moon Pink Floyd 1973-03-01 Progressive rock
    ALBUM_1 TRACK_1 track_title run_length    
    Speak to Me PT1M13S    
    ALBUM_1 TRACK_2 track_title run_length    
    Breathe PT2M43S    
    ALBUM_1 TRACK_3 track_title run_length    
    On the Run PT3M36S    
    ...
    ALBUM_2 INFO album_title album_artiest release_date genre
    The Wall Pink Floyd 1979-11-30 Progressive rock
    ALBUM_2 TRACK_1 track_title run_length    
    In the Flesh? PT3M20S    
    ...
    This table uses a composite primary key, (parition_key, sort_key), to identify each item.
  • The key ("ALBUM_1", "INFO") identifies ALBUM_1's metadata.
  • The key ("ALBUM_1", "TRACK_1") identifies ALBUM_1's first track.
  • This table stores tracks belonging to the same album together and sorts them by the track number. The application needs only one request to DynamoDB to get the album and its tracks.
    aws dynamodb query \
        --table-name music_library_items \
        --key-conditions '{ 
            "PK": { 
                "ComparisonOperator": "EQ",
                "AttributeValueList": [ { "S": "ALBUM_1" } ]
            } 
        }'
    Why Tempest?
    For locality, we smashed together several entity types in the same table. This improves performance! But it breaks type safety in DynamoDBMapper.
    DynamoDBMapper API
    // NOTE: This is not Tempest! It is an example used for comparison.
    @DynamoDBTable(tableName = "music_library_items")
    class MusicLibraryItem {
      // All Items.
      @DynamoDBHashKey
      var partition_key: String? = null
      @DynamoDBRangeKey
      var sort_key: String? = null
    
      // AlbumInfo.
      @DynamoDBAttribute
      var album_title: String? = null
      @DynamoDBAttribute
      var album_artist: String? = null
      @DynamoDBAttribute
      var release_date: String? = null
      @DynamoDBAttribute
      var genre: String? = null
    
      // AlbumTrack.
      @DynamoDBAttribute
      var track_title: String? = null
      @DynamoDBAttribute
      var run_length: String? = null
    }
    Note that MusicLibraryItem is a union type of all the entity types: AlbumInfo and AlbumTrack. Because all of its attributes are nullable and mutable, code that interacts with it is brittle and error prone.
    Tempest API
    Tempest restores maintainability without losing locality. It lets you declare strongly-typed key and item classes for each logical type in the domain layer.
    data class AlbumInfo(
      @Attribute(name = "partition_key")
      val album_token: String,
      val album_title: String,
      val album_artist: String,
      val release_date: String,
      val genre_name: String
    ) {
      @Attribute(prefix = "INFO_")
      val sort_key: String = ""
    
      data class Key(
        val album_token: String
      ) {
        val sort_key: String = ""
      }
    }
    
    data class AlbumTrack(
      @Attribute(name = "partition_key")
      val album_token: String,
      @Attribute(name = "sort_key", prefix = "TRACK_")
      val track_token: String,
      val track_title: String,
      val run_length: String
    ) {
      data class Key(
        val album_token: String,
        val track_token: String
      )
    }
    You build business logic with logical types. Tempest handles mapping them to the underlying persistence type.
    interface MusicLibraryTable : LogicalTable<MusicLibraryItem> {
      val albumInfo: InlineView<AlbumInfo.Key, AlbumInfo>
      val albumTracks: InlineView<AlbumTrack.Key, AlbumTrack>
    }
    
    private val musicLibrary: MusicLibraryTable
    
    // Load.
    fun getAlbumTitle(albumToken: String): String? {
      val key = AlbumInfo.Key(albumToken)
      val albumInfo = musicLibrary.albumInfo.load(key) ?: return null
      return albumInfo.album_title
    }
    
    // Update.
    fun addAlbumTrack(
      albumToken: String, 
      track_token: String, 
      track_title: String, 
      run_length: String
    ) {
      val newAlbumTrack = AlbumTrack(albumToken, track_token, track_title, run_length)
      musicLibrary.albumTracks.save(newAlbumTrack)
    } 
    
    // Query.
    fun getAlbumTrackTitles(albumToken: String): List<String> {
      val page = musicLibrary.albumTracks.query(
        keyCondition = BeginsWith(AlbumTrack.Key(albumToken))
      )
      return page.contents.map { it.track_title }
    }
    Get Tempest
    See the project website for documentation and APIs.
    For AWS SDK 1.x:
    implementation "app.cash.tempest:tempest:1.5.2"
    For AWS SDK 2.x:
    implementation "app.cash.tempest:tempest2:1.5.2"

    14

    This website collects cookies to deliver better user experience

    Efficient DynamoDB in Kotlin with Tempest