Open source software is more than just code. It’s also about the people who create, use, and contribute to it. One of the key factors to the rise of open source software is the power of community. Community is not only a source of users, but also a source of contributors, advocates, evangelists, and potential employees. Community is what gives open source software feedback, validation, trust, and loyalty. It is what makes open source software better and more relevant. But how do you build and maintain a strong and engaged community around your open source product or service? And how do you leverage the community to grow your business and achieve your goals?

Ian Tien (Mattermost’s CEO) explained how building software for free attracted a lot of people who wanted to help the project because they were getting a lot of value out of it. He said: “A lot of people just tend to help you [when you’re building software for free], right? Because they’re getting a lot of value out of it. And it’s not like you’re hard selling them […] You’re building something you love. People respond to the value.”

One of the ways that open source communities demonstrate their power is by organizing themselves and organizing meetups and events to connect with each other and promote their project. Saurav Pathak, Co-Founder and Chief Product Officer of Bagisto, revealed the project’s strategy of creating a global network of ambassadors who organized local events around the product. Bagisto recruited ambassadors from different countries and gave them the resources and support to host local gatherings and spread the word about the product. According to Saurav:  “I think it’s a huge success for us that some people are taking initiative and they are actually working around Bagisto and doing these small meetups.”

It becomes evident that building a community around your open source project is a great way to attract contributors, users and supporters. But how do you know if your community is healthy and thriving? How do you measure the impact and value of your community efforts? That’s where metrics come in.

Metrics are quantitative and qualitative indicators that help you track and evaluate the performance and progress of your community. There is no one-size-fits-all way to measure an open source community. Different projects may have different goals, audiences, and challenges. Georg Link, Director of Sales at Bitergia, told us: “Depending on the use case, there are different metrics that I would look at. But the common denominator is the number of contributions and the activity in the project.”

What Investors Want to See in Your Community Metrics:

Investing in open source projects is not the same as investing in traditional software companies. Therefore, investors need to understand the dynamics and health of these communities, as well as the value they generate for the project and its users. 

Unlike traditional software companies, open source projects do not have standard metrics such as revenue, profit, or market share. Instead, they have different indicators that reflect their community activity, engagement, growth, diversity, quality, and impact. These indicators can help investors assess the viability, sustainability, and scalability of open source projects, as well as their alignment with their own goals and values. 

Jana Iris, investor at TQ Ventures, mentions the metrics she has followed and suggests various metrics such as the amount of activities, the number of posts per week, responses to GitHub issues. She explains that at her previous company, HashiCorp, the goal was to educate and familiarize people with the product, which resulted in signups to their existing learning platform. Jana said: “Of course, each team had its own specific number, like tracking core contributor sentiment over time, tracking ambassador engagement over time, tracking Slack engagement or GitHub discussions. So there's various ways you can look at the health of the project.” 

Joseph Jacks, founder and investor at OSS Capital, has a different approach to evaluating open source communities. He believes that the most important metric to look at from an investor’s perspective is growth rate, more specifically the rate of growth of external contributors. He explains that external contributors are the ones who drive adoption, innovation, and value creation for open source projects. However, not all growth metrics are equally interesting or meaningful for Jacks. He says that some metrics, such as the rate of growth of GitHub stars, are not very useful or reliable indicators of community health or value. According to Jacks: “I think that, on its own, it doesn't give you a strong proxy for actual usage and actual adoption.”

Jacks also emphasizes the importance of building relationships with the founders of open source projects as humans, not just as potential investments. He explains: “We, in fact, had to actually have a reasonable sense of conviction well before the creators themselves even had the conviction, which makes it even harder.”

The Most Important Community Metrics to Track, According to the Experts:

If you are running or contributing to an open source community, you may wonder what are the most important metrics to measure its success and impact. However, there is no one-size-fits-all answer to this question. Depending on your background, goals and role, you may have different metrics that you prefer or different methods of measuring them. For instance, someone running a foundation may look at metrics very differently than someone looking at it from a sales and marketing perspective. Similarly, someone working in developer relations may have a different perspective than someone working in engineering or product management. In this section, we will compare and contrast the views of these different stakeholders and explore the most important community metrics to track for each of them.

Maxim Wheatly, Head of Global Marketing and Sales at Merico, identifies three metrics that he uses to evaluate the health and performance of their community: “For me, those metrics are Slack growth, new contributors, and retention. They give me insights into discoverability, community growth, and user engagement.“

Although it is beneficial to have clear and specific metrics to measure your community’s success, some experts in the field suggest that numbers are not everything. They argue that there are other aspects of community health and value that cannot be easily quantified or captured by standard indicators.

When it comes to measuring the health and brand recognizability of an open source project, there are several metrics that go beyond numbers and statistics. Kaj Arnö, CEO of the MariaDB Foundation, highlights that by listening to the anecdotes of your friends, family, colleagues, or peers, you can learn about their awareness and understanding of your project. He also advocates mixing both these anecdotes and more concrete metrics to get a more well-rounded picture saying: “Looking at Jira and observing how contributions grow and how they're handled, including votes on Jira items, allows us to understand what people truly desire. Unlike those stars, they do offer valuable assistance.”

Depending on the role and the organization, DevRel professionals may have different goals and objectives. Therefore, it is important to define and measure the right metrics that align with the specific goals and objectives of each role and organization. Patrick McFadin, Developer Advocate at DataStax, revealed that the most meaningful metrics for him revolve mostly around engagement and community contributions such as the number of contributed articles and posted use cases. He says: “After all, you're creating content for people to use, so measuring their usage is important. These metrics can be quite helpful in determining what resonates and what doesn't with the community. By analyzing such data, we can make informed decisions.” 

Move Beyond Vanity Metrics and Measure What Really Matters for Your Community:

However, not all metrics are equally meaningful or useful for every project. Some metrics may be misleading or irrelevant, such as GitHub stars, which do not reflect the actual quality, impact, or sustainability of a project.

GitHub stars are a simple way to show appreciation or interest in a project, but they do not indicate how many people are actually using, contributing to, or benefiting from it. GitHub stars can also be easily manipulated or inflated by bots, campaigns, or incentives. Therefore, relying too much on such so-called “vanity metrics'' as a measure of success can be counterproductive or deceptive. 

Joe Karlsson, Developer Advocate at Tinybird, suggested that it’s also important to have in mind the “why” of tracking specific metrics and checking if it aligns with the business goals. About this, he said: “I think it affects the quality or what you're doing with that content too. You could do something that's salesy and really push hard to convert people, but it may turn people away or push them to sign up for GitHub stars and forget that your community isn't growing”

Conclusion

In this blog, we have explored how to measure the success of your open source community. We have seen that community is a key factor for the growth and impact of open source software, and that metrics can help you track and evaluate your community’s health and value. We have heard from experts in the field who shared their insights and experiences on how they use metrics to build and maintain strong and engaged communities around their open source projects.

We hope that this blog has given you some useful tips and ideas on how to measure your own open source community’s success. Remember that metrics are not an end in themselves, but a means to an end. They are tools that can help you understand your community better, identify its strengths and weaknesses, and improve its performance and impact. However, metrics are not everything. They cannot capture the human and social aspects of community, such as trust, loyalty, passion, and collaboration. These are the intangible qualities that make open source communities unique and powerful. Therefore, while you use metrics to measure your community’s success, don’t forget to also celebrate your community’s spirit.