20
GitHub Stargazers over time using Google Charts
Announcing… Planetarium! A way to generate a chart of GitHub Stargazers over time using Google Charts.
Tracking the popularity of your GitHub repository over time is something that might appeal to you. ‘Stargazers’ are a public list of GitHub users that have starred (essentially bookmarked) your repository and is a rough indicator of engagement. People either want to use your code, or at least not forget the link to follow up on it another time.
Whilst it’s not public, the GitHub API exposes the date and time of when each person starred your repository. This then makes it possible to aggregate these stars into a pretty graph which is a cool way to see how your code’s popularity has increased over time.
Whilst ruminating over the surprising popularity of our recent Serverless Static WordPress on AWS module, I was wondering how to get my own graph like others I’d seen elsewhere. The source of these seemed to be this Starcharts repo written in Go. Whilst this was pretty cool, it seemed to have a few limitations and lacked configurability. Plus it’s written in Go, which I’m definitely never going to learn.
My first thought was to get the Stargazing data myself and plot it in a Google sheet. But how? Wrestling with the output of an API to get data into your desired format is always a bit frustrating, but my previous experience with Steampipe, which lets you query a variety of APIs, including AWS and GitHub, in glorious SQL code, made this extremely easy. After writing up a simple query, I was able to get the data I wanted, aggregated over time in a single result. Awesome!
So I put it in my Google sheet, and I generate a graph manually, and that looks fine – but then suddenly I realise that Google Charts is actually a service and I could potentially generate this dynamically. Was it possible?
Well what was initially a bit of curious enquiry turned into a delirious 2am mission to not sleep until I could get dynamic GitHub query + chart generation as a single PNG file working. Man was it not easy!
Google Charts is nice enough, but you can only render charts from an HTML page with embedded Javascript with your data encoded within the page. None of this is dynamic and at first glance this doesn’t seem particularly easy to automate. It is possible to get your data imported using an external data source, but it needs to be available as a queriable URL. That means I’d have to build my own API to return some data. It’s possible, but I just want to plug in some basic CSV data here!
So down another rabbit hole I go… Steampipe to query GitHub and deposit a nicely-formatted CSV file into my local workspace. I then use jquery.csv.js to load the CSV file into Google Charts with Javascript. But… you can’t do this using a file:// path in a local browser due to CORS security controls, so the HTML has to be loaded from a real webserver….
I want this to be a quick end-to-end file generation process, so I create an extremely minimal not-even webserver in Python that runs just long enough to process 3 requests before shutting itself down. But loading the page in the browser just renders the image as an Image Data URI… this means getting the image I want is not as simple as just curling a URL, because there is no URL!
Various server-side methods of loading the page won’t work, because they critically need a Javascript interpreter, just like a real browser…. hmmm. What about headless Chrome and Selenium? I don’t want to install any of this locally, so I’ll need that in a Docker container please – luckily one exists.
So after a couple of hours of punching some Python code and working out the worst of the bugs, I finally compress it all into a single shell script that’ll grab the latest stargazer source data, then run a docker container with selenium so I can use a Python library to load the page and automatically save the image before depositing it back into the local working directory.
It might be a terrible solution because I’ve missed a much more obvious, easier way of doing this, but in any case I remain amazed at how you can set your sights on a silly, pointless goal and then spend a few hours grinding away in code and many, many searches to finally get it working in the way you imaged.
My next iteration of this will be to bundle it into a Lambda function and have it run on a schedule, to update an image dynamically loaded into the README of my desired GitHub repo! Watch this space for that.
Full instructions and more details are available on the Planetarium GitHub page. I hope you find it useful!
--
Pete Wilcock is a 9x AWS Certified DevOps Consultant, AWS Community Builder, and Technical Writer for TechToSpeech. If I’m not possibly losing my mind and definitely my social life buried in some project, you can find me on LinkedIn, Twitter, GitHub, or my personal site.
20