Measuring Downloads of Anything You Distribute

This article was originally posted on

Hackernoon

Back in March we first launched Scarf Gateway with support for Docker/OCI containers. No matter which registry your containers are hosted on, you can now host all of them through your own central endpoint, and have better visibility into that download traffic than any other registry on the market. All without ever having access to personally identifiable information or invading the privacy of your users.

Today, we've expanded the gateway's capabilities beyond serving containers, to support downloads for any kind of file. Whether you distribute archives like {% c-line %}.zip{% c-line-end %}, or  {% c-line %}.tar.gz{% c-line-end %}, text files like {% c-line %}JSON{% c-line-end %}, or even raw executables, they can now be served via Scarf, from anywhere on the internet. Your files can be anywhere - maybe you upload artifacts to GitHub Releases, or S3. Scarf sits right on top of whatever your workflow and stack might be, and augments it with advanced analytics you won't find on whatever site or tool you're using today.

A Quick Example

GitHub Releases is a common way to publish and distribute artifacts for your project. Unfortunately, it does not offer much in the way of observability into how your artifacts are downloaded, but that's exactly where Scarf can help. I'll give an example for one of my own personal packages, curl-runnings. On the releases page, curl-runnings distributes binaries for Mac and for Debian-flavored Linux, and until now, I didn't know much about its downloads.

An example download URL for one of these releases might look like:

https://github.com/aviaviavi/curl-runnings/releases/download/0.16.0/curl-runnings-0.16.0-mac.tar.gz

Factoring in the different artifacts being distributed for each release, and the associated version numbers for each release, we have a URL template of the form:

https://github.com/aviaviavi/curl-runnings/releases/download/{version}/curl-runnings-{version}-{platform}.tar.gz

Scarf Gateway provides an easy way to map any new incoming URL template we'd like to this original URL template above for our release artifacts. I chose to use my own domain, {% c-line %}downloads.avi.press{% c-line-end %}, and pick a new incoming URL path {% c-line %}/curl-runnings-{version}-{platform}.tar.gz{% c-line-end %}. Now, a URL like:

https://downloads.avi.press/curl-runnings-0.15.0-linux.tar.gz

Will resolve to:

https://github.com/aviaviavi/curl-runnings/releases/download/0.15.0/curl-runnings-0.15.0-linux.tar.gz

The final step is using the new links anywhere these archives are linked and referenced. On GitHub Releases, this just means adding the new links in the release description.

Point users to your new Scarf-powered download links

When a user clicks the link and the download is initiated, Scarf will log a download with the variable values {% c-line %}version=0.16.0{% c-line-end %} and {% c-line %}platform=mac{% c-line-end %}. Histograms for all variables are available within the Scarf Dashboard.

Scarf automatically reports your downloads for all variables in your URL templates

With Scarf, curl-runnings has fully measurable and customizable download URLs!

Give it a Try

You can think of Scarf Gateway as a powerful, programmable link shortener with advanced analytics for anything you distribute for direct download. If you've ever wanted to measure which download links on your site were performing better, or which marketing campaign was converting most effectively into actual downloads of your software, you now have a new tool in your toolbox.

Scarf Gateway is free for all open-source projects. You can get started here, or head to the docs to learn more. We're excited to see what you build!

Latest blog posts

Tools and strategies modern teams need to help their companies grow.

Stefano Maffulli: An Exploration on Standards for Open Source Packaging and Distribution

Stefano Maffulli: An Exploration on Standards for Open Source Packaging and Distribution

Scarf Sessions is a new stream where we have conversations with people shaping the landscape in open source and open source sustainability. This post will give a recap of the conversation Scarf CEO, Avi Press and I had with our guest Stefano Maffulli.
Using OSS Usage Data to Sell your Company

Using OSS Usage Data to Sell your Company

Learn how Nestybox used Scarf to gather better project insights and provide accurate data during their recent acquisition.
A Different Approach to Measuring Open Source Community Health

A Different Approach to Measuring Open Source Community Health

Community is important to the success of open source software. To understand and grow a community, project founders and maintainers need visibility into various technical, social, and even financial metrics. But what metrics should we be using?