Published
May 30, 2022
This article was originally posted on
Hackernoon
Community is important to the success of open source software. The health of a community can be measured in different ways depending on the stakeholders involved. To understand and grow a community, project founders and maintainers need visibility into various technical, social, and even financial metrics. These metrics are important because they can help projects form data driven decisions. But what metrics should we be using?
This post is inspired by several discussions I have seen around the failings of current open source metrics and the tools used to measure them. This post will discuss what metrics we currently use and what I believe we should be using instead.
On LinkedIn’s publishing platform Pulse, William Tracz wrote about Open Source vanity metrics and the problem with using them to measure a project's success. Tracz lamented they could not currently see a better way to gather valuable data about open source projects.
What are these current metrics?
Like Tracz mentions, many of the metrics open source maintainers and contributors rely on are considered “vanity metrics”. This term does not mean these metrics are not useful but they often do not give a full picture of a communities health.
Examples of vanity metrics in open source are:
and more.
Often these are metrics founders and maintainers care about as they provide a way to quickly show “success” but may not actually help when it comes to important business outcomes and decisions
Orbit is a tool you can integrate with GitHub to see metics like:
and more.
As an open source community manager I used Orbit to quickly track new comments from contributors and maintainers so I could respond in a timely manner. I also used it to see when community members were no longer active. This helped me reach out and get feedback.
CHAOSS is a Linux Foundation project focused on creating analytics and metrics to help define community health.
Common Room is a growth platform that focuses on a business’ community. The platform helps companies connect their community engagement, product usage, and customer data through the use of artificial intelligence.
Bitergia is an analytics tool for software development teams, open source program offices, and InnerSource programs. Open source projects can use this tool to improve decision making and reporting by analyzing software development community, activity, and performance of open source projects.
Merico provides tools to help engineering teams understand and optimize their productivity and code quality.
“Tracking different types of metrics is essential for free and open source communities. Metrics give project insights into specific efforts and help get a feel of the community’s general perception. For that, tools that can pull data from various sources and develop a visualization of this data will help projects make informed decisions.” - Linux.com Editorial Staff, Measuring the Health of Open Source Communities
In the article quoted above and in William Tracz’s article the need for a tool that can pull more valuable data is apparent. But what data should projects be looking at and what tool could they use?
What if you could see a project’s impact, identify potential customers, and understand/showcase growth without sacrificing your end-user’s privacy?
Scarf Gateway helps you understand how users are interacting with your open-source project at every step of their journey
Instead of answering questions like “how many followers do we have”, Scarf’s data digs deeper in to a project’s community health and answers questions like:
We can also have visibility into how OSS software is distributed.
Instead of wondering who is using our software, Scarf’s data answers questions like:
Scarf can also help uncover community metrics like your projects discoverability, usage, churn, and commercial reliance. Community Managers can use this data to proactively connect with users and grow a healthy and vibrant OSS community.
This data compliments the metrics you can get from tools like Orbit and answers questions like:
For a project’s documentation it can answer:
This type of rich usage data moves beyond the typical vanity metrics the open source community has typically relied on. While those metrics can tell you a lot about a project, they do not give the full picture of a community's health.
With the addition of tools like Scarf Gateway, maintainers now have the power to make more informed decisions. When maintainers have hard data around who, where, and why companies and individuals are using their software they can better prioritize features, patches, releases and more.
Tools and strategies modern teams need to help their companies grow.
.png)
.png)
.png)
.png)
.png)
.png)
.png)
.png)
.png)
.png)
.png)
