From operating systems to libraries, frameworks to middleware, applications, and beyond, open-source projects across different domains find themselves caring about all sorts of different metrics. Some domains of OSS will care about some metrics far more than others will. Language-level packages might pay close attention to their number of downstream dependents or contributors, while application developers may have more usage and end-user community-based interests.
What metric should all projects care about? What metric is both generally relevant and is also in some way tied to how much value a project is delivering?
Downloads will almost always matter. Whether it's SDK downloads from a package manager, a pull of a containerized OS image, or a simple GET request to a tarball or webpage, any value that a piece of OSS can provide always starts with a user downloading that software so they can either run it, use it, modify it, etc. Even when the software is being hosted in cloud environments, the cloud provider must download the software in order to productize it. Downloads may not be a clear or complete picture of the story, but they are a fundamental action that anyone must take with the software to get any benefit from it. If you want to better understand your project’s growth and the journey users take to adopt it, you won’t have a complete picture without visibility into your downloads.
Your project can be of value to someone who never stars it on GitHub, never contributes back, never opens an issue, never joins your community, or never tweets about you. All they need to do is use it; anyone who uses it will have to download it.
Download numbers are only sometimes a straightforward metric to make sense of, let alone glean useful insights from. They are often noisy and opaque. If you got 1000 downloads last month, was that from 1000 people, or just 1? (Our data suggests your 1000 downloads came from about 60 people on average). Just because someone downloads your software, it in no way means they are using it, or will ever use it. However, someone who starred your repo on GitHub is closer to using it than someone who has your software on their laptop. Stars represent nothing more than interest, it’s too high up in your funnel to measure your project. To use a piece of OSS, downloading it is a necessary early step.
Download numbers are certainly not a panacea for OSS metrics, but they are a very important source of helpful information about the impact of your work. They can give clues to where the project is being used, how it's being used, and by which organizations. They can be used to infer user engagement ("When we put out a new release, how quickly do our users upgrade? How often do we see repeat downloads?") and commercial opportunities if your project is commercially backed ("Which companies did we see download for the first time this month?").
Downloads are crucial to understanding your project’s funnel, as they can be used as an important conversion milestone. Downloads can help you measure the effectiveness of your documentation and website ("Which page of our docs/site had the strongest or weakest correlations with continuing on to download?"), your marketing efforts ("Did we see a bump in downloads after we published that last blog post?"), or local community engagement ("Did we see a spike in downloads from the area that we just gave a talk in?"). To optimize your project’s growth and adoption, downloads are the first place you should look.
When paired with other metrics more specific to your project, downloads are an important piece of the picture of how your project is doing, and how users step through your funnel. From taking the first step with your project to relying on it heavily enough that they engage with you and your community — or perhaps your company’s product.
For those that are commercializing OSS, you should know that for better or for worse, companies have been funded and sold on their download metrics. They are an essential piece of pitching your open-source business and making the case for the value delivered.
As analytics tools like Scarf become increasingly sophisticated at gleaning insights from OSS download traffic, maintainers can better understand their work's impact, make data-driven decisions, and build more sustainable and healthy projects.
