How to build and run a free link tracker using VS Code, Java, GitHub, MongoDB and Azure

Many developers are interested in learning how to build and run software on cloud platforms, but are wary of the potential for hidden costs and bill shock.

Often times this bill shock come from not understanding how consumption-based services work, and by being lulled into a false sense of security by the relatively tiny amount per-request / per-MB many services charge. This is all well-and-good until you build something unexpectedly popular that explodes over night and ramps up you bill.

However, most of us will actually never face this dilema and I think it’s a shame that people are limiting themselves to learn something new based on what might happen, not what will happen. Having said this, let’s look at how we can remove some uncertainty when building with Azure.

Building cost-safe solutions

Most of the solutions I’ve built cover the last 18 months easily fit into the “runs for the price of a coffee (at Sydney prices) or less” category, but I am always looking for ways to do better. To this end I am going to try and find ways for you to learn with Azure without the worry that you might need a second mortgage.

The first thing I would recommend doing is looking at the Azure Free Tier and understanding what services it contains and what their limits are, with a particular focus on what happens when you exceed the Free Tier offer. You should revisit periodically as services are added or modified over time – as an example Cosmos DB recently increased the Request Units (RUs) and storage available under their free offer.

Also, consider what you want to do or learn with the platform. If you are completely new to Azure and want to make use of the time-limited financial credits and 12 month increased free service tier, make sure you have goals in mind, otherwise you’ll waste your benefits. I wrote about this in 2018, so it’s worth a read before starting.

Finally, always setup Budget alerts for PAYG subscriptions you pay for. This won’t stop exponential growth happening, but at least you will know about it sooner than your next bill and can act accordingly to limit your expenses.

Our demo scenario

For this post I will be building a simple REST API that can be used to create and track URL shortlinks (think bit.ly). These can be used on blogs like mine to track outbound link clicks.

If I want track any outbound link (say when I reference a GitHub repository), I create a shortlink for the destination, use that shortlink in my blog in place of the actual destination and when a blog visitor clicks on the link their request first hits my shortlink system before being redirected to the actual destination. The shortlink code logs the request and I can report on it later.

As I like to mix things up I’m going to build the solution using Java Spring Boot and MongoDB.

Free local development and deployment

If I’m going to challenge myself to run services for free, I might as well extend that to the entire lifecycle of my application. To that end I’m going to be using the following to help me build and debug my solution locally.

Once I’m done writing my solution, I want somewhere to publish my code and to then deploy it to Azure, so for that I’ll be using GitHub and GitHub Actions.

Free hosting in Azure

As a developer I hate managing infrastructure. If it requires a Virtual Machine, you’re doing it wrong! So, for hosting our solution I am going to use Azure App Service and Azure Cosmos DB configured with MongoDB API support. Both these services offer a compelling free tier:

  • App Service: F1 Free Tier – not designed for prod workloads, but guarantees you stay free. When your app consumes more than 60 minutes of CPU time in a day the web app will be stopped and callers will receive a 503 (Unavailable) message.
  • Cosmos DB: Free Tier – you can run some good small solutions on this tier. The 1,000 Request Units (RUs) and 25 GB is generous and should suit most use cases. If you generate calls that exceed the RU limit you’ll receive a 429 (Too Many Requests) message and will need to handle in your calling code. You can manage the storage growth by using Time-To-Live (TTL) settings to ensure you don’t exceed the limits, though 25 GB is a lot of documents!

Let’s build our app!

As a starting point, connect to your MongoDB instance and make sure you have an empty database available. I called my database “bloglinks”, but you can use any name you want. Copy the connection URI and database name for use later on.

You can also create an empty repository on GitHub where the application source code will be stored when you are finished and ready to deploy to Azure.

Now, using Visual Studio Code, open an empty folder on your computer and then bring up the command palette (View > Command Palette) and select Spring Initialzr by starting to type ‘spring’. Select “Create a Maven Project…”

Next up let’s select the Spring Boot version. I tend to favour stable releases, but the choice is yours. Note your versions will likely look different to the below one depending on when you read this blog.

Make the project language Java.

Now set up the necessary details to generate your package name.

Set Packaging Type to ‘Jar’ and Java version to ’11’.

Finally, let’s pull in the Spring Dependencies that will help us get going quickly with our solution. For our application we’ll use Spring Web, Spring Data MongoDB and Spring Security.

OK, so now we have a basic Spring Boot web application scaffold which we can build on top of. Rather than spend more time on pulling this together I am going to direct you to the source code for a pre-built version of the application which you can use as the basis of yours.

Make sure to pay attention to the application.properties.prod file. You can use this locally on your machine (rename it application.properties) to manage configuration for basic authentication and connection to MongoDB. It’s important to ensure you add this file to your gitignore when committing to GitHub though, especially if you use a remote MongoDB instance.

Once you’ve completed coding up the solution you will find that the Spring Data MongoDB library will automatically create the necessary Collections for you in your Database. We will use these two Collection names when we create placeholders in Azure in our next step.

Setting up Azure

We will use the Azure CLI to complete these steps, so make sure you have it installed and that you have logged into the subscription you want to deploy to.

First we’ll create a Resource Group which can contain all our services. You can choose the location that suits you.

az group create --name MyResourceGroup --location westus2

Create a free Azure App Service Plan.

az appservice plan create \
    --resource-group MyResourceGroup \
    --name MyAppsPlan \
    --is-linux \
    --sku FREE

Create Web App using the Plan. We will pre-configure the Web App with the right Java environment (Java SE web server 8 with Java 11).

az webapp create \
    --name myuniquewebappid123 \ 
    --resource-group MyResourceGroup \
    --runtime "JAVA|8-java11" \
    --plan MyAppsPlan

Create free tier Cosmos DB configurated with the MongoDB API v4.0.

az cosmosdb create \
    --name MyCosmosAccount \
    --resource-group MyResourcegroup \
    --enable-free-tier true \
    --kind MongoDB \
    --server-version 4.0 \
    --default-consistency-level "Session"

Now we can create our Database and Collections in Cosmos DB.

# Create 'bloglinks' Database
az cosmosdb mongodb database create \
     --account-name MyCosmosAccount \
     --resource-group MyResourcegroup \
     --name bloglinks 

# Create 'shortLink' Collection
az cosmosdb mongodb collection create \
     --account-name MyCosmosAccount \
     --resource-group MyResourcegroup \
     --database-name bloglinks \
     --name shortLink

# Create 'linkClick' Collection
az cosmosdb mongodb collection create \
     --account-name MyCosmosAccount \
     --resource-group MyResourcegroup \
     --database-name bloglinks \
     --name linkClick \
     --idx '[{"key":{"keys": ["_ts"]},"options":{"expireAfterSeconds": 5184000}}]'

Now, you might look at that last Collection create statement and wonder what I am doing here? Well, remember earlier I mentioned we wanted a way to ensure we don’t hit our storage limit (even though that’s unlikely to ever happen… but why find out?!), well this is how I will manage it this scenario.

Cosmos DB has a great Time-To-Live (TTL) feature which can be used to expire documents, meaning they are removed from result sets and ultimately purged from storage. Best of all? No cost involved!!

If you want to use this feature with the MongoDB API you need to use Mongo Indexes over the Cosmos-internal “_ts” document property to drive this feature. Based on the official Microsoft documentation you can define these indexes when creating a Collection (as above) or separately later using a MongoDB client to create the index. I’ve set the expiry at 60 days as I should ideally of processed the collection data by this point, and if not… that’s on me!!

As a last step, let’s grab the MongoDB connection URI for our Cosmos DB we can use in our configuration below.

az cosmosdb keys list \
     --name MyCosmosAccount \
     --resource-group MyResourcegroup \
     --type connection-strings

OK, so at this stage we have a web server and a MongoDB database ready for our application to be deployed.

Deploying and configuring our web application

The final piece of the puzzle is to deploy and configure our web application.

Let’s start with the configuration. We have four configuration elements we need in order for our web application to work:

  • spring.security.user.name
  • spring.security.user.password
  • spring.data.mongodb.database
  • spring.data.mongodb.uri

Locally we have been holding the values in our application.properties file, but we don’t want to push that file to GitHub, so make sure when you publish the solution to GitHub that you exclude this file.

Azure Application Service provides a nice neat way to provide these four values via App Settings which we can deploy as follows. Convert each application.properties placeholder into an equivalent with underscores instead of dots.

az webapp config appsettings set \
    --resource-group MyResourcegroup \
    --name myuniquewebappid123 \
    --settings SPRING_SECURITY_USER_NAME=youruser \
SPRING_SECURITY_USER_PASSWORD=5ecureP4ssword \
SPRING_DATA_MONGODB_DATABASE=bloglinks \
SPRING_DATA_MONGODB_URI=mongodb://your_cosmos_uri/

Finally, let’s configure the GitHub Action to deploy the application for us.

az webapp deployment github-actions add \
    --resource-group MyResourcegroup \
    --name myuniquewebappid123 \
    --repo "youruser/your-repo" \
    --runtime "JAVA|8-java11" \
    --login-with-github

You will be prompted to log into GitHub at https://github.com/login/device using a specific ID, and after completed you should find that there is now a new folder in your repository which contains the workflow which will build and deploy your solution!

After a few minutes you find that a workflow kicks off and your application is now deployed and ready to serve requests!

Testing it out

Before we wrap up, let’s test out our deployed solution. The easiest way to do this is to install Thunderclient for VS Code and then open the collection I’ve included in the sample app repository. Make sure to update the hostname to match the one you used in Azure, and that you set the basic authentication username and password to match yours too.

When you use the POST call to create a new shortlink you will receive a “201 Created” HTTP response code and the details for the created entry are in HTTP response header as the ‘location’ field. This is shown below. If you try to create a link that already exists you will receive a “200 OK” HTTP response code and the shortlink data will be returned in the response body.

Wrapping up

We’ve done a lot in this post, but I’m not 100% done yet. If you’ve run all the requests you will notice that there is a field called “visitors” that has a value of zero. We need to build a solution that will populate this field for use periodically, ideally in a zero-cost fashion as well. In a future post I’ll take a look at how we can do this.

In the meantime… happy days! 😎

19

Categories:

JavaMongodbGithub

Tags:

AzureJavaMongodbGithub