Measuring Downloads of Anything You Distribute

Published

September 7, 2021

This article was originally posted on

Hackernoon

Back in March we first launched Scarf Gateway with support for Docker/OCI containers. No matter which registry your containers are hosted on, you can now host all of them through your own central endpoint, and have better visibility into that download traffic than any other registry on the market. All without ever having access to personally identifiable information or invading the privacy of your users.

Today, we've expanded the gateway's capabilities beyond serving containers, to support downloads for any kind of file. Whether you distribute archives like {% c-line %}.zip{% c-line-end %}, or  {% c-line %}.tar.gz{% c-line-end %}, text files like {% c-line %}JSON{% c-line-end %}, or even raw executables, they can now be served via Scarf, from anywhere on the internet. Your files can be anywhere - maybe you upload artifacts to GitHub Releases, or S3. Scarf sits right on top of whatever your workflow and stack might be, and augments it with advanced analytics you won't find on whatever site or tool you're using today.

A Quick Example

GitHub Releases is a common way to publish and distribute artifacts for your project. Unfortunately, it does not offer much in the way of observability into how your artifacts are downloaded, but that's exactly where Scarf can help. I'll give an example for one of my own personal packages, curl-runnings. On the releases page, curl-runnings distributes binaries for Mac and for Debian-flavored Linux, and until now, I didn't know much about its downloads.

An example download URL for one of these releases might look like:

https://github.com/aviaviavi/curl-runnings/releases/download/0.16.0/curl-runnings-0.16.0-mac.tar.gz

Factoring in the different artifacts being distributed for each release, and the associated version numbers for each release, we have a URL template of the form:

https://github.com/aviaviavi/curl-runnings/releases/download/{version}/curl-runnings-{version}-{platform}.tar.gz

Scarf Gateway provides an easy way to map any new incoming URL template we'd like to this original URL template above for our release artifacts. I chose to use my own domain, {% c-line %}downloads.avi.press{% c-line-end %}, and pick a new incoming URL path {% c-line %}/curl-runnings-{version}-{platform}.tar.gz{% c-line-end %}. Now, a URL like:

https://downloads.avi.press/curl-runnings-0.15.0-linux.tar.gz

Will resolve to:

https://github.com/aviaviavi/curl-runnings/releases/download/0.15.0/curl-runnings-0.15.0-linux.tar.gz

The final step is using the new links anywhere these archives are linked and referenced. On GitHub Releases, this just means adding the new links in the release description.

Point users to your new Scarf-powered download links

When a user clicks the link and the download is initiated, Scarf will log a download with the variable values {% c-line %}version=0.16.0{% c-line-end %} and {% c-line %}platform=mac{% c-line-end %}. Histograms for all variables are available within the Scarf Dashboard.

Scarf automatically reports your downloads for all variables in your URL templates

With Scarf, curl-runnings has fully measurable and customizable download URLs!

Give it a Try

You can think of Scarf Gateway as a powerful, programmable link shortener with advanced analytics for anything you distribute for direct download. If you've ever wanted to measure which download links on your site were performing better, or which marketing campaign was converting most effectively into actual downloads of your software, you now have a new tool in your toolbox.

Scarf Gateway is free for all open-source projects. You can get started here, or head to the docs to learn more. We're excited to see what you build!

Latest blog posts

Tools and strategies modern teams need to help their companies grow.

The Most Neglected and Overlooked Open Source Metric: Production Users

The Most Neglected and Overlooked Open Source Metric: Production Users

Everyone wants a larger open source user base, but very few people effectively measure its growth. Let’s discuss why.
Switching Container Registries With Zero Downtime

Switching Container Registries With Zero Downtime

You can use the open source Scarf Gateway to switch hosting providers, container registries, or repositories without impacting end users in the future.
Understanding Tech Layoffs and the Economy’s Impact on Open Source

Understanding Tech Layoffs and the Economy’s Impact on Open Source

What is driving all this tech layoffs? , What is their impact on the open source software industry? We will walk through all the potential reasons from an economic downturn, herd mentality, excessive borrowing and spending due to low interest rates, and growth at all costs as the main reasons behind the layoffs. Companies can continue to grow in this tight economic market if they are focused on optimizing efficiency and sustaining the right growth.