By clicking “Accept all”, you agree to the storing of cookies on your device to enhance site navigation, analyze site usage, and assist in our marketing efforts. View our Cookie Policy for more information.
Open source software is more than just code. It’s also about the people who create, use, and contribute to it. One of the key factors to the rise of open source software is the power of community. Community is not only a source of users, but also a source of contributors, advocates, evangelists, and potential employees. Community is what gives open source software feedback, validation, trust, and loyalty. It is what makes open source software better and more relevant. But how do you build and maintain a strong and engaged community around your open source product or service? And how do you leverage the community to grow your business and achieve your goals?
Open source software thrives on the feedback, contributions, and support of its community.
Ian Tien (Mattermost’s CEO) explained how building software for free attracted a lot of people who wanted to help the project because they were getting a lot of value out of it. He said: “A lot of people just tend to help you [when you’re building software for free], right? Because they’re getting a lot of value out of it. And it’s not like you’re hard selling them […] You’re building something you love. People respond to the value.”
One of the ways that open source communities demonstrate their power is by organizing themselves and organizing meetups and events to connect with each other and promote their project. Saurav Pathak, Co-Founder and Chief Product Officer of Bagisto, revealed the project’s strategy of creating a global network of ambassadors who organized local events around the product. Bagisto recruited ambassadors from different countries and gave them the resources and support to host local gatherings and spread the word about the product. According to Saurav: “I think it’s a huge success for us that some people are taking initiative and they are actually working around Bagisto and doing these small meetups.”
It becomes evident that building a community around your open source project is a great way to attract contributors, users and supporters. But how do you know if your community is healthy and thriving? How do you measure the impact and value of your community efforts? That’s where metrics come in.
Metrics are quantitative and qualitative indicators that help you track and evaluate the performance and progress of your community. There is no one-size-fits-all way to measure an open source community. Different projects may have different goals, audiences, and challenges. Georg Link, Director of Sales at Bitergia, told us: “Depending on the use case, there are different metrics that I would look at. But the common denominator is the number of contributions and the activity in the project.”
What Investors Want to See in Your Community Metrics:
Investing in open source projects is not the same as investing in traditional software companies. Therefore, investors need to understand the dynamics and health of these communities, as well as the value they generate for the project and its users.
Unlike traditional software companies, open source projects do not have standard metrics such as revenue, profit, or market share. Instead, they have different indicators that reflect their community activity, engagement, growth, diversity, quality, and impact. These indicators can help investors assess the viability, sustainability, and scalability of open source projects, as well as their alignment with their own goals and values.
Jana Iris, investor at TQ Ventures, mentions the metrics she has followed and suggests various metrics such as the amount of activities, the number of posts per week, responses to GitHub issues. She explains that at her previous company, HashiCorp, the goal was to educate and familiarize people with the product, which resulted in signups to their existing learning platform. Jana said: “Of course, each team had its own specific number, like tracking core contributor sentiment over time, tracking ambassador engagement over time, tracking Slack engagement or GitHub discussions. So there's various ways you can look at the health of the project.”
Joseph Jacks, founder and investor at OSS Capital, has a different approach to evaluating open source communities. He believes that the most important metric to look at from an investor’s perspective is growth rate, more specifically the rate of growth of external contributors. He explains that external contributors are the ones who drive adoption, innovation, and value creation for open source projects. However, not all growth metrics are equally interesting or meaningful for Jacks. He says that some metrics, such as the rate of growth of GitHub stars, are not very useful or reliable indicators of community health or value. According to Jacks: “I think that, on its own, it doesn't give you a strong proxy for actual usage and actual adoption.”
Jacks also emphasizes the importance of building relationships with the founders of open source projects as humans, not just as potential investments. He explains: “We, in fact, had to actually have a reasonable sense of conviction well before the creators themselves even had the conviction, which makes it even harder.”
Investing in open source projects requires a different approach than investing in traditional software companies.
The Most Important Community Metrics to Track, According to the Experts:
If you are running or contributing to an open source community, you may wonder what are the most important metrics to measure its success and impact. However, there is no one-size-fits-all answer to this question. Depending on your background, goals and role, you may have different metrics that you prefer or different methods of measuring them. For instance, someone running a foundation may look at metrics very differently than someone looking at it from a sales and marketing perspective. Similarly, someone working in developer relations may have a different perspective than someone working in engineering or product management. In this section, we will compare and contrast the views of these different stakeholders and explore the most important community metrics to track for each of them.
Maxim Wheatly, Head of Global Marketing and Sales at Merico, identifies three metrics that he uses to evaluate the health and performance of their community: “For me, those metrics are Slack growth, new contributors, and retention. They give me insights into discoverability, community growth, and user engagement.“
Although it is beneficial to have clear and specific metrics to measure your community’s success, some experts in the field suggest that numbers are not everything. They argue that there are other aspects of community health and value that cannot be easily quantified or captured by standard indicators.
When it comes to measuring the health and brand recognizability of an open source project, there are several metrics that go beyond numbers and statistics. Kaj Arnö, CEO of the MariaDB Foundation, highlights that by listening to the anecdotes of your friends, family, colleagues, or peers, you can learn about their awareness and understanding of your project. He also advocates mixing both these anecdotes and more concrete metrics to get a more well-rounded picture saying: “Looking at Jira and observing how contributions grow and how they're handled, including votes on Jira items, allows us to understand what people truly desire. Unlike those stars, they do offer valuable assistance.”
Depending on the role and the organization, DevRel professionals may have different goals and objectives. Therefore, it is important to define and measure the right metrics that align with the specific goals and objectives of each role and organization. Patrick McFadin, Developer Advocate at DataStax, revealed that the most meaningful metrics for him revolve mostly around engagement and community contributions such as the number of contributed articles and posted use cases. He says: “After all, you're creating content for people to use, so measuring their usage is important. These metrics can be quite helpful in determining what resonates and what doesn't with the community. By analyzing such data, we can make informed decisions.”
Experts say that metrics are helpful but not enough to measure your community’s success, as some aspects are too qualitative or complex to quantify.
Move Beyond Vanity Metrics and Measure What Really Matters for Your Community:
However, not all metrics are equally meaningful or useful for every project. Some metrics may be misleading or irrelevant, such as GitHub stars, which do not reflect the actual quality, impact, or sustainability of a project.
GitHub stars are a simple way to show appreciation or interest in a project, but they do not indicate how many people are actually using, contributing to, or benefiting from it. GitHub stars can also be easily manipulated or inflated by bots, campaigns, or incentives. Therefore, relying too much on such so-called “vanity metrics'' as a measure of success can be counterproductive or deceptive.
Joe Karlsson, Developer Advocate at Tinybird, suggested that it’s also important to have in mind the “why” of tracking specific metrics and checking if it aligns with the business goals. About this, he said: “I think it affects the quality or what you're doing with that content too. You could do something that's salesy and really push hard to convert people, but it may turn people away or push them to sign up for GitHub stars and forget that your community isn't growing”
Conclusion
In this blog, we have explored how to measure the success of your open source community. We have seen that community is a key factor for the growth and impact of open source software, and that metrics can help you track and evaluate your community’s health and value. We have heard from experts in the field who shared their insights and experiences on how they use metrics to build and maintain strong and engaged communities around their open source projects.
We hope that this blog has given you some useful tips and ideas on how to measure your own open source community’s success. Remember that metrics are not an end in themselves, but a means to an end. They are tools that can help you understand your community better, identify its strengths and weaknesses, and improve its performance and impact. However, metrics are not everything. They cannot capture the human and social aspects of community, such as trust, loyalty, passion, and collaboration. These are the intangible qualities that make open source communities unique and powerful. Therefore, while you use metrics to measure your community’s success, don’t forget to also celebrate your community’s spirit.
This is some text inside of a div block.
Lorem ipsum dolor sit amet, consectetur adipiscing elit. Suspendisse varius enim in eros elementum tristique. Duis cursus, mi quis viverra ornare, eros dolor interdum nulla, ut commodo diam libero vitae erat. Aenean faucibus nibh et justo cursus id rutrum lorem imperdiet. Nunc ut sem vitae risus tristique posuere.
Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur.
Open source software is more than just code. It’s also about the people who create, use, and contribute to it. One of the key factors to the rise of open source software is the power of community. Community is not only a source of users, but also a source of contributors, advocates, evangelists, and potential employees. Community is what gives open source software feedback, validation, trust, and loyalty. It is what makes open source software better and more relevant. But how do you build and maintain a strong and engaged community around your open source product or service? And how do you leverage the community to grow your business and achieve your goals?
Open source software thrives on the feedback, contributions, and support of its community.
Ian Tien (Mattermost’s CEO) explained how building software for free attracted a lot of people who wanted to help the project because they were getting a lot of value out of it. He said: “A lot of people just tend to help you [when you’re building software for free], right? Because they’re getting a lot of value out of it. And it’s not like you’re hard selling them […] You’re building something you love. People respond to the value.”
One of the ways that open source communities demonstrate their power is by organizing themselves and organizing meetups and events to connect with each other and promote their project. Saurav Pathak, Co-Founder and Chief Product Officer of Bagisto, revealed the project’s strategy of creating a global network of ambassadors who organized local events around the product. Bagisto recruited ambassadors from different countries and gave them the resources and support to host local gatherings and spread the word about the product. According to Saurav: “I think it’s a huge success for us that some people are taking initiative and they are actually working around Bagisto and doing these small meetups.”
It becomes evident that building a community around your open source project is a great way to attract contributors, users and supporters. But how do you know if your community is healthy and thriving? How do you measure the impact and value of your community efforts? That’s where metrics come in.
Metrics are quantitative and qualitative indicators that help you track and evaluate the performance and progress of your community. There is no one-size-fits-all way to measure an open source community. Different projects may have different goals, audiences, and challenges. Georg Link, Director of Sales at Bitergia, told us: “Depending on the use case, there are different metrics that I would look at. But the common denominator is the number of contributions and the activity in the project.”
What Investors Want to See in Your Community Metrics:
Investing in open source projects is not the same as investing in traditional software companies. Therefore, investors need to understand the dynamics and health of these communities, as well as the value they generate for the project and its users.
Unlike traditional software companies, open source projects do not have standard metrics such as revenue, profit, or market share. Instead, they have different indicators that reflect their community activity, engagement, growth, diversity, quality, and impact. These indicators can help investors assess the viability, sustainability, and scalability of open source projects, as well as their alignment with their own goals and values.
Jana Iris, investor at TQ Ventures, mentions the metrics she has followed and suggests various metrics such as the amount of activities, the number of posts per week, responses to GitHub issues. She explains that at her previous company, HashiCorp, the goal was to educate and familiarize people with the product, which resulted in signups to their existing learning platform. Jana said: “Of course, each team had its own specific number, like tracking core contributor sentiment over time, tracking ambassador engagement over time, tracking Slack engagement or GitHub discussions. So there's various ways you can look at the health of the project.”
Joseph Jacks, founder and investor at OSS Capital, has a different approach to evaluating open source communities. He believes that the most important metric to look at from an investor’s perspective is growth rate, more specifically the rate of growth of external contributors. He explains that external contributors are the ones who drive adoption, innovation, and value creation for open source projects. However, not all growth metrics are equally interesting or meaningful for Jacks. He says that some metrics, such as the rate of growth of GitHub stars, are not very useful or reliable indicators of community health or value. According to Jacks: “I think that, on its own, it doesn't give you a strong proxy for actual usage and actual adoption.”
Jacks also emphasizes the importance of building relationships with the founders of open source projects as humans, not just as potential investments. He explains: “We, in fact, had to actually have a reasonable sense of conviction well before the creators themselves even had the conviction, which makes it even harder.”
Investing in open source projects requires a different approach than investing in traditional software companies.
The Most Important Community Metrics to Track, According to the Experts:
If you are running or contributing to an open source community, you may wonder what are the most important metrics to measure its success and impact. However, there is no one-size-fits-all answer to this question. Depending on your background, goals and role, you may have different metrics that you prefer or different methods of measuring them. For instance, someone running a foundation may look at metrics very differently than someone looking at it from a sales and marketing perspective. Similarly, someone working in developer relations may have a different perspective than someone working in engineering or product management. In this section, we will compare and contrast the views of these different stakeholders and explore the most important community metrics to track for each of them.
Maxim Wheatly, Head of Global Marketing and Sales at Merico, identifies three metrics that he uses to evaluate the health and performance of their community: “For me, those metrics are Slack growth, new contributors, and retention. They give me insights into discoverability, community growth, and user engagement.“
Although it is beneficial to have clear and specific metrics to measure your community’s success, some experts in the field suggest that numbers are not everything. They argue that there are other aspects of community health and value that cannot be easily quantified or captured by standard indicators.
When it comes to measuring the health and brand recognizability of an open source project, there are several metrics that go beyond numbers and statistics. Kaj Arnö, CEO of the MariaDB Foundation, highlights that by listening to the anecdotes of your friends, family, colleagues, or peers, you can learn about their awareness and understanding of your project. He also advocates mixing both these anecdotes and more concrete metrics to get a more well-rounded picture saying: “Looking at Jira and observing how contributions grow and how they're handled, including votes on Jira items, allows us to understand what people truly desire. Unlike those stars, they do offer valuable assistance.”
Depending on the role and the organization, DevRel professionals may have different goals and objectives. Therefore, it is important to define and measure the right metrics that align with the specific goals and objectives of each role and organization. Patrick McFadin, Developer Advocate at DataStax, revealed that the most meaningful metrics for him revolve mostly around engagement and community contributions such as the number of contributed articles and posted use cases. He says: “After all, you're creating content for people to use, so measuring their usage is important. These metrics can be quite helpful in determining what resonates and what doesn't with the community. By analyzing such data, we can make informed decisions.”
Experts say that metrics are helpful but not enough to measure your community’s success, as some aspects are too qualitative or complex to quantify.
Move Beyond Vanity Metrics and Measure What Really Matters for Your Community:
However, not all metrics are equally meaningful or useful for every project. Some metrics may be misleading or irrelevant, such as GitHub stars, which do not reflect the actual quality, impact, or sustainability of a project.
GitHub stars are a simple way to show appreciation or interest in a project, but they do not indicate how many people are actually using, contributing to, or benefiting from it. GitHub stars can also be easily manipulated or inflated by bots, campaigns, or incentives. Therefore, relying too much on such so-called “vanity metrics'' as a measure of success can be counterproductive or deceptive.
Joe Karlsson, Developer Advocate at Tinybird, suggested that it’s also important to have in mind the “why” of tracking specific metrics and checking if it aligns with the business goals. About this, he said: “I think it affects the quality or what you're doing with that content too. You could do something that's salesy and really push hard to convert people, but it may turn people away or push them to sign up for GitHub stars and forget that your community isn't growing”
Conclusion
In this blog, we have explored how to measure the success of your open source community. We have seen that community is a key factor for the growth and impact of open source software, and that metrics can help you track and evaluate your community’s health and value. We have heard from experts in the field who shared their insights and experiences on how they use metrics to build and maintain strong and engaged communities around their open source projects.
We hope that this blog has given you some useful tips and ideas on how to measure your own open source community’s success. Remember that metrics are not an end in themselves, but a means to an end. They are tools that can help you understand your community better, identify its strengths and weaknesses, and improve its performance and impact. However, metrics are not everything. They cannot capture the human and social aspects of community, such as trust, loyalty, passion, and collaboration. These are the intangible qualities that make open source communities unique and powerful. Therefore, while you use metrics to measure your community’s success, don’t forget to also celebrate your community’s spirit.
Latest blog posts
Tools and strategies modern teams need to help their companies grow.
Keeping your Salesforce accounts in sync with Scarf’s company data gives your team a clearer picture of how organizations are engaging with your open-source project.
Companies offering open source are pouring resources into the project, but it remains a black hole for go-to-market teams—high adoption, but no clear path to conversion. With Scarf, company-level usage data is available while maintaining privacy. The native integration with Salesforce and Hubspot, brings your open source users from passive usage to pipeline acceleration.
Open source projects generate a treasure trove of data (downloads, doc views, project usage), but most open source companies aren’t tapping into it. Here’s how Scarf helps us, and how it can help you do the same with your marketing.
RisingWave, an open-source streaming database, is reshaping real-time data processing with its SQL-powered platform. Before Scarf, RisingWave relied on a Metabase dashboard to track anonymous telemetry. Scarf gave RisingWave the insights they were missing. By providing visibility into company-level engagement, Scarf is helping them identify, track and re-engage with companies already using RisingWave.
Scarf provided Weaviate with the clarity and actionable insights they needed. By integrating Scarf into their open-source project, Weaviate unlocked real-time developer signals, such as download volumes, referral sources, and organizational trends.
Monthly Tracked Companies (MTCs) are the organizations actively engaging with your open source project. This includes downloads of your packages, views of your documentation, and any other type of interaction with your software. Scarf identifies and surfaces these organizations so you can better understand your audience and prioritize outreach.
As 2024 wraps up, we’ve been reflecting on everything that wouldn’t have been possible without the Scarf community and our amazing customers. It’s been a big year for Scarf—and for the open source projects we’re proud to support.
The Scarf Summit brought together open source industry leaders to explore how open source usage signals are shaping the future of commercial open source companies. We were joined by Soham Maniar, Director of RevOps at Weaviate and Kevin White, Head of Marketing at Common Room, to expand on leveraging open source usage data for sales and marketing campaigns.
This playbook will guide you through the steps to set up and embed a Scarf Pixel on your documentation pages, README files, or any other web properties associated with your project, in this case we will focus specifically on documentation.
Today, the most commonly accepted metrics for open source adoption and growth are heavily focused on the contributors and community (the idea is healthy contributions should equate to healthy adoption). While these are useful metrics, they are only part of the picture. This guide is built for those at open-source-based companies who are responsible for growth and adoption.
We’ve got some exciting news: Scarf just launched a powerful, native integration with Salesforce, bringing Scarf’s rich open source usage data directly into your CRM. No more bouncing between tools or setting up S3 data exports—you can now get all the insights you need where you already do your work.
Scarf, a platform designed to provide open-source projects with deeper insights into their users and usage patterns, was the answer ARMO needed. By integrating Scarf into Kubescape, ARMO was able to regain visibility into which company has been using Kubescape, filling the gap left after their CNCF contribution.
The foundation of Scarf company tracking is IP Address attribution. Our Company Tracking algorithm considers confidence and reputation scores from multiple sources to provide what we believe to be the best matching data in the industry. In a nutshell, Match Feedback allows you to fix and fine-tune your company matches.
We're thrilled to announce that Scarf has successfully completed the SOC 2 Type 2 examination! This might sound like legal jargon at first glance, but let’s break down what this means for us, our users, and the open-source community as a whole.
Scarf helps you unlock the full potential of your open source project by collecting valuable usage data in three key ways: Scarf Packages, in-app telemetry, and tracking pixels. In this post, we’ll break down each of these powerful tools and show you how to use them to optimize your open source strategy.
Exporting data tracked by Scarf is essential for analytics, reporting, and integration with other tools. Scarf adds open-source usage metrics to the data you already collect, giving you a fuller picture of how your project is used. This helps you monitor trends, measure impact, and make better data-driven decisions.
In this playbook, you’ll learn how to integrate Scarf into an Apache Software Foundation project. It details how the Preset team implemented Scarf in their Apache Superset project, as shared during our first-ever Scarf Summit on July 16th, 2024.
Prisma turned to Scarf for a monthly Strategic Insights Report. By integrating Scarf into various parts of their web and software delivery infrastructure, Prisma now knows relevant details about their users in terms of company size, industry, location and much more.
Implementing telemetry in your open source project helps you determine whether people are testing your software and continuing its use over time. Such insights not only confirm if the developed software meets users' needs but also helps identify which versions are being adopted and which might be vulnerable to the latest bugs or other issues.
This playbook will walk you through setting up Scarf to get a clearer picture of how people are interacting with your open-source project. You’ll learn how to create and use Scarf Pixels, track open source project documentation views, measure engagement across social media, and more.
CopilotKit implemented Scarf to gain visibility into their open-source community. By adding Scarf to their documentation, they could see which companies were actively engaging with their resources, providing valuable insights into potential leads and customer segments.
Tracking downloads of your open-source projects is key to understanding user engagement. With Scarf, you can see which businesses are using your project, which versions are popular, which platforms are being targeted, and more. This playbook will show you how to set up Scarf to monitor your project’s downloads.
On July 16th, we hosted our first-ever Scarf Summit, celebrating analytics for open source and the significant improvements we’ve made to the Scarf platform. In case you missed it, here’s a recap of all the key updates shared by our Engineering Leader, Aaron Porter.
In this episode of the Haskell Interlude Podcast, Joachim Breitner and Andreas Löh sit down with Avi Press, the founder of Scarf, to discuss his journey with Haskell, the telemetry landscape in open source software, and the technical as well as operational challenges of building a startup with Haskell at its core.
Scarf Basic and Premium tiers have long had the ability to sort their open source usage data by company, domain, events, last seen, and funnel stage. But our customers have been wanting more. Now you can hyper target by combining region, tech stack, and funnel stage, making outreach as refined and low friction as possible.
Understanding open source user engagements and usage is obscured by a lack of actionable data, a result of its inherent openness and anonymity. Embracing a data-driven approach to open source projects helps them not only grow, but also understand the keys to their success, benefiting everyone involved.
As an open source company, Garden knew how hard it was going to be to get usage data. Adding Scarf for analytics on open source downloads turned anonymous numbers into company names. Using Scarf’s privacy-first analytics also helped Garden to know what kind of companies were using their OSS and where they were located.
Once Heroic started using Scarf, they learned that they were even more popular than they thought they were. Using Scarf, they were able to determine where, by country, their users were downloading from, and how many per day.
Any LF project maintainer can use Scarf without needing any further approval from the foundation. Scarf is offering all LF projects free accounts with a few additional features over our base free version. LF projects will get usage data like docs, downloads, and page views with unlimited free seat licenses and data retention.
Union is an open source first company. It uses Scarf to drive their DevRel strategy and improve their open source project. It also uses Scarf to power its consultative sales approach to help customers where it makes sense. Union has been successfully leveraging Scarf funnel analysis to shape the product to better fit the market so that they can focus on ensuring that companies can get value from Flyte sooner.
In this latest episode of "Hacking Open Source Business," Avi Press and Matt Yonkovit sit down with Adam Jacob, the co-founder of Chef and current CEO of System Initiative. With a rich history in the open-source world and numerous thought-provoking opinions, Adam delves into the intricacies of open-source commercialization, offering valuable insights and alternative strategies to the commonly held Open Core model.
Smallstep wanted to understand the impact of their open-source project on enterprise adoption of their commercial security solutions. Smallstep uses Scarf to better understand user interactions and software usage, providing insights into its user base and potential customer segments as an important signal for commercial use.
Diagrid was founded in 2022 by the creators of the popular Dapr open source project. Making data-driven decisions for a commercial company built on an open source project that had no real concrete data, was a real challenge. Diagrid translated Scarf data into valuable insights for marketing and product development of their commercial product.
When we approached the project of building Scarf, we turned to our favorite language: Haskell. Little did we know, this decision would shape our story in more ways than one.
Unstructured had so much usage of their open source, but so little data. Prior to Scarf, they mostly had GitHub information for things like downloads and stars. It was difficult to separate the good signal from the noise without any specific information that would help them to better target this large and growing open source user base or data to influence their product roadmap.
It’s happening! Scarf is part of the Common Room Signal Partners program. Soon, you will be able to integrate your Scarf data into your Common Room platform for a more complete view of all of your user signals.
We are thrilled to announce that we have successfully completed a Type 1 System and Organization Controls 2 (SOC 2) examination for our Scarf Platform service as of January 31, 2024.
When Scarf emerged back in 2019, many people expressed skepticism that usage analytics would ever be tolerated in the open source world. 5 years later, Scarf has shown this once solidified cultural norm can indeed change. Learn how Scarf's journey mirrors a broader shift in open source culture and why embracing usage analytics could shape the future of open software development.
Apache Superset is an open-source modern data exploration and visualization platform that makes it easy for users of all skill sets to explore and visualize their data. We spoke with Maxime Beauchemin, founder & CEO of Preset, and the original creator of both Apache Superset and Apache Airflow, who shared with us Superset's experience using Scarf.
Haskell, a cutting-edge programming language rooted in pure functionality, boasts static typing, type inference, and lazy evaluation. The language's ongoing evolution is bolstered by a diverse array of organizations, including the Haskell.org committee. This committee strategically leveraged the Scarf solution for testing purposes.
We’re pleased to share a final recap of the latest Scarf updates for December and 2023 as a whole. Join us in this last edition of our 2023 newsletters.
In the open source ecosystem, user behaviors are diverse and conversion tracking poses unique challenges frequently leaving traditional marketing strategies insufficient. Recognizing this gap, we are excited to introduce a brand new way for businesses to make sense of this opaque and noisy signal – Open Source Qualified Leads (OQLs).
In recent years, a notable development in the open source landscape is the growing number of large corporations considering the transition from open source licenses to more restrictive models like the Business Source License (BSL). This trend raises further questions about the sustainability and future of open source projects, particularly when large players alter their approach.
A recent release of Scarf added the ability to track and report on custom URL parameters. If you are looking to gain more intelligence around how you open source users interact with your project and download your software using link parameters in key situations can reveal interesting and helpful trends that can help you grow your user base and unlock open source qualified leads.
In the ever-evolving landscape of open source software, data collection has become a hot-button issue. As the open source community grows and software becomes increasingly integral to our daily lives, concerns about data collection ethics have emerged.
In today's fast-paced tech world, the Developer Relations (DevRel) role has moved from the periphery to the center stage. Companies, irrespective of their size, are now seriously considering the worth of having a dedicated DevRel team. But, how do you quantify the success or failure of such an effort? What metrics should companies use? This post dives deep into understanding the commercial Return on Investment (ROI) of DevRel.
Monetizing open source software is a challenging task, but it can also be highly rewarding. Unlike traditional software, you're essentially competing against a free version of your product. So, how do you sell something that is inherently free?
In the dynamic realm of community management, marketing, and developer relations, success depends upon more than just attracting attention. It's about fostering meaningful relationships, nurturing engagement, and amplifying your community's impact.
This guidebook shows you how to implement a call-home functionality or telemetry within your open-source software while at the same time being transparent and respectful of your users data. Let's explore how to build a minimal, privacy-focused call home functionality using a simple version check and Scarf.
Many open source contributors are reluctant or skeptical about metrics. They think metrics are overrated, irrelevant, or even harmful to their projects and communities. But in this blog post, we argue that metrics are essential for making better decisions, improving the experience for users and contributors, and demonstrating the impact and value of your open source work. We also share some tips and examples from OSPOs and DevRel teams on how to choose and use metrics effectively.
Many open-source developers rely on GitHub as their primary documentation source. But this can be a costly mistake that can affect your project’s success and adoption. In this blog, we’ll explain why you need to build your own docs site and how to do it easily and effectively.
Open source projects and companies need data to grow and enhance their performance. However, many open source leaders and communities overlook or reject metrics and depend on intuition, relationships, or imitation. Data can help you spot problems, opportunities, and false positives in growth strategies. In this blog post, Matt Yonkovit shows you why data is important for open source success and how it can offer insights and guidance for open source projects to reach their goals and make better decisions.
Open source software continues to be a vital part of enterprise operations in Q2 2023, as more and more companies adopt open source solutions for their business needs. In this blog post, we will examine the state of open source usage in Q2 2023 and the trends that are shaping the future of open source.
DevRel is a vital function for any organization that wants to engage with the developer community and grow its user base. However, there is no one-size-fits-all solution for where to place DevRel within the organizational structure. In this blog post, we explore three common strategies for DevRel placement: marketing, product, and hybrid. We discuss the advantages and challenges of each strategy, and provide some tips on how to decide which one is best for your organization and goals.
In the open source industry, identifying and engaging users is a major challenge. Many users download software from third-party platforms that do not share user data with the software company. Gating content behind a login or an email form can help, but it can also alienate potential users who value their privacy and convenience. In this blog post, we explore the pros and cons of gating content in the open source industry, and we offer an alternative solution that can help you identify and connect with your users without compromising your content.
Open source software depends on the power of its community. But how do you know if your community is healthy and thriving? In this blog, you will learn how to use metrics to track and evaluate your community’s activity, engagement, growth, diversity, quality, and impact. You will hear from founders, DevRel experts, and investors who share their best practices and tips on how to measure and improve your community’s performance and value.
Learn how to overcome the challenges of open source software marketing and turn anonymous data into qualified leads. In this blog post, we’ll show you how to use download data, web traffic, and documentation views to identify potential customers and grow your sales pipeline. Discover how to track downloads, website traffic and documentation views with Scarf Gateway and the Scarf Tracking Pixel.
This blog post outlines ten common mistakes made by founders of open source startups, from failing to ask the right questions to neglecting the standardization of key metrics. By offering guidance on how to avoid these pitfalls, it provides a roadmap to successfully commercializing open source projects.
Many people believe that making money from open source projects is an arduous or even impossible task. However, with the right strategies it is possible to build a sustainable business while keeping the spirit of open source intact. By evaluating the market fit and commercial viability of an open source project before considering funding and monetization, one can realistically begin to explore the financial potential of an open source project. Here's how to do it.
This blog emphasizes the importance of a comprehensive approach to lead generation in the open source software space. Amid the challenges of anonymous usage and privacy regulations, strategies focusing on download activity, community engagement, and web traffic can maximize lead identification. Employing lead scoring and maintaining a list of active software users can further enhance sales outcomes in this unique market.
Here at Scarf, we've developed a solution to help open source projects and businesses gain more insight into their users and their download traffic - Scarf Gateway. Here's how it works.
We are thrilled to announce our latest partnership with Clearbit (https://clearbit.com/). This collaboration will offer Scarf users and customers an enriched array of data about their user base, significantly enhancing the quality of information you already value from Scarf.
The popularity of open source software is not in doubt, but little concrete public data exists beyond human-generated surveys on adoption usage. In this blog post, we will explore the state of open source usage in Q1 2023 and the data illustrating how open source is becoming an increasingly important part of enterprise operations.
The success of DevRel (Developer Relations) and community efforts in open source can be challenging to measure, as there is often a disconnect between the goals and expectations of the community and the business. This blog post discusses the challenges of measuring the success of DevRel and community efforts in open source.
Successful open source projects don't always translate into successful open source businesses. However, by focusing on building a kick-ass product, raising awareness, making the product easier to use, and fostering a strong open source community, you can set the stage for converting users into paying customers.
You can use the open source Scarf Gateway to switch hosting providers, container registries, or repositories without impacting end users in the future.
What is driving all this tech layoffs? , What is their impact on the open source software industry? We will walk through all the potential reasons from an economic downturn, herd mentality, excessive borrowing and spending due to low interest rates, and growth at all costs as the main reasons behind the layoffs. Companies can continue to grow in this tight economic market if they are focused on optimizing efficiency and sustaining the right growth.
At the All Things Open conference, Emily Omier, a seasoned positioning consultant, sat down with Avi Press (Founder and CEO, Scarf) and Matt Yonkovit (The HOSS, Scarf) to discuss how to message, position, and validate your open source product on The Hacking Open Source Business Podcast. You can watch the full episode below or continue reading for a recap.
On the Hacking Open Source Business podcast, Joseph Jacks aka JJ (Founder, OSS Capital) joins Avi Press (Founder and CEO, Scarf) and Matt Yonkovit (The HOSS, Scarf) to share what you need to know before starting a commercial open source software (COSS) company and how you can set yourself and your project apart in a way that attracts investor funding. As an investor who exclusively focuses on open source startups, JJ provides a VC perspective on what he looks for when evaluating investment opportunities.
On The Hacking Open Source Business podcast, CEO Chris Molozian and Head of Developer Relations Gabriel Pene at Heroic Labs elaborate on their usage and shift to open source and how it accelerated their adoption.
In this recap of the first episode of the Hacking Open Source Business Podcast, co-hosts Matt Yonkovit and Avi Press, Scarf Founder and CEO, dig into a recent controversy that highlights the challenges open source projects face trying to create sustainable revenue streams to support a business or a non-profit that funds the project’s growth.
Scarf Sessions is a new stream where we have conversations with people shaping the landscape in open source and open source sustainability. This post will give a recap of the conversation Scarf CEO, Avi Press and I had with our guest Stefano Maffulli.
Community is important to the success of open source software. To understand and grow a community, project founders and maintainers need visibility into various technical, social, and even financial metrics. But what metrics should we be using?
Should Python eggs be deprecated in favor of wheels? What does the data show? This post explores how the right data can make decisions like this easier for maintainers and Open Source organizations.
Keeping your Salesforce accounts in sync with Scarf’s company data gives your team a clearer picture of how organizations are engaging with your open-source project.
Companies offering open source are pouring resources into the project, but it remains a black hole for go-to-market teams—high adoption, but no clear path to conversion. With Scarf, company-level usage data is available while maintaining privacy. The native integration with Salesforce and Hubspot, brings your open source users from passive usage to pipeline acceleration.
Open source projects generate a treasure trove of data (downloads, doc views, project usage), but most open source companies aren’t tapping into it. Here’s how Scarf helps us, and how it can help you do the same with your marketing.
Open source software is more than just code. It’s also about the people who create, use, and contribute to it. One of the key factors to the rise of open source software is the power of community. Community is not only a source of users, but also a source of contributors, advocates, evangelists, and potential employees. Community is what gives open source software feedback, validation, trust, and loyalty. It is what makes open source software better and more relevant. But how do you build and maintain a strong and engaged community around your open source product or service? And how do you leverage the community to grow your business and achieve your goals?
Open source software thrives on the feedback, contributions, and support of its community.
Ian Tien (Mattermost’s CEO) explained how building software for free attracted a lot of people who wanted to help the project because they were getting a lot of value out of it. He said: “A lot of people just tend to help you [when you’re building software for free], right? Because they’re getting a lot of value out of it. And it’s not like you’re hard selling them […] You’re building something you love. People respond to the value.”
One of the ways that open source communities demonstrate their power is by organizing themselves and organizing meetups and events to connect with each other and promote their project. Saurav Pathak, Co-Founder and Chief Product Officer of Bagisto, revealed the project’s strategy of creating a global network of ambassadors who organized local events around the product. Bagisto recruited ambassadors from different countries and gave them the resources and support to host local gatherings and spread the word about the product. According to Saurav: “I think it’s a huge success for us that some people are taking initiative and they are actually working around Bagisto and doing these small meetups.”
It becomes evident that building a community around your open source project is a great way to attract contributors, users and supporters. But how do you know if your community is healthy and thriving? How do you measure the impact and value of your community efforts? That’s where metrics come in.
Metrics are quantitative and qualitative indicators that help you track and evaluate the performance and progress of your community. There is no one-size-fits-all way to measure an open source community. Different projects may have different goals, audiences, and challenges. Georg Link, Director of Sales at Bitergia, told us: “Depending on the use case, there are different metrics that I would look at. But the common denominator is the number of contributions and the activity in the project.”
What Investors Want to See in Your Community Metrics:
Investing in open source projects is not the same as investing in traditional software companies. Therefore, investors need to understand the dynamics and health of these communities, as well as the value they generate for the project and its users.
Unlike traditional software companies, open source projects do not have standard metrics such as revenue, profit, or market share. Instead, they have different indicators that reflect their community activity, engagement, growth, diversity, quality, and impact. These indicators can help investors assess the viability, sustainability, and scalability of open source projects, as well as their alignment with their own goals and values.
Jana Iris, investor at TQ Ventures, mentions the metrics she has followed and suggests various metrics such as the amount of activities, the number of posts per week, responses to GitHub issues. She explains that at her previous company, HashiCorp, the goal was to educate and familiarize people with the product, which resulted in signups to their existing learning platform. Jana said: “Of course, each team had its own specific number, like tracking core contributor sentiment over time, tracking ambassador engagement over time, tracking Slack engagement or GitHub discussions. So there's various ways you can look at the health of the project.”
Joseph Jacks, founder and investor at OSS Capital, has a different approach to evaluating open source communities. He believes that the most important metric to look at from an investor’s perspective is growth rate, more specifically the rate of growth of external contributors. He explains that external contributors are the ones who drive adoption, innovation, and value creation for open source projects. However, not all growth metrics are equally interesting or meaningful for Jacks. He says that some metrics, such as the rate of growth of GitHub stars, are not very useful or reliable indicators of community health or value. According to Jacks: “I think that, on its own, it doesn't give you a strong proxy for actual usage and actual adoption.”
Jacks also emphasizes the importance of building relationships with the founders of open source projects as humans, not just as potential investments. He explains: “We, in fact, had to actually have a reasonable sense of conviction well before the creators themselves even had the conviction, which makes it even harder.”
Investing in open source projects requires a different approach than investing in traditional software companies.
The Most Important Community Metrics to Track, According to the Experts:
If you are running or contributing to an open source community, you may wonder what are the most important metrics to measure its success and impact. However, there is no one-size-fits-all answer to this question. Depending on your background, goals and role, you may have different metrics that you prefer or different methods of measuring them. For instance, someone running a foundation may look at metrics very differently than someone looking at it from a sales and marketing perspective. Similarly, someone working in developer relations may have a different perspective than someone working in engineering or product management. In this section, we will compare and contrast the views of these different stakeholders and explore the most important community metrics to track for each of them.
Maxim Wheatly, Head of Global Marketing and Sales at Merico, identifies three metrics that he uses to evaluate the health and performance of their community: “For me, those metrics are Slack growth, new contributors, and retention. They give me insights into discoverability, community growth, and user engagement.“
Although it is beneficial to have clear and specific metrics to measure your community’s success, some experts in the field suggest that numbers are not everything. They argue that there are other aspects of community health and value that cannot be easily quantified or captured by standard indicators.
When it comes to measuring the health and brand recognizability of an open source project, there are several metrics that go beyond numbers and statistics. Kaj Arnö, CEO of the MariaDB Foundation, highlights that by listening to the anecdotes of your friends, family, colleagues, or peers, you can learn about their awareness and understanding of your project. He also advocates mixing both these anecdotes and more concrete metrics to get a more well-rounded picture saying: “Looking at Jira and observing how contributions grow and how they're handled, including votes on Jira items, allows us to understand what people truly desire. Unlike those stars, they do offer valuable assistance.”
Depending on the role and the organization, DevRel professionals may have different goals and objectives. Therefore, it is important to define and measure the right metrics that align with the specific goals and objectives of each role and organization. Patrick McFadin, Developer Advocate at DataStax, revealed that the most meaningful metrics for him revolve mostly around engagement and community contributions such as the number of contributed articles and posted use cases. He says: “After all, you're creating content for people to use, so measuring their usage is important. These metrics can be quite helpful in determining what resonates and what doesn't with the community. By analyzing such data, we can make informed decisions.”
Experts say that metrics are helpful but not enough to measure your community’s success, as some aspects are too qualitative or complex to quantify.
Move Beyond Vanity Metrics and Measure What Really Matters for Your Community:
However, not all metrics are equally meaningful or useful for every project. Some metrics may be misleading or irrelevant, such as GitHub stars, which do not reflect the actual quality, impact, or sustainability of a project.
GitHub stars are a simple way to show appreciation or interest in a project, but they do not indicate how many people are actually using, contributing to, or benefiting from it. GitHub stars can also be easily manipulated or inflated by bots, campaigns, or incentives. Therefore, relying too much on such so-called “vanity metrics'' as a measure of success can be counterproductive or deceptive.
Joe Karlsson, Developer Advocate at Tinybird, suggested that it’s also important to have in mind the “why” of tracking specific metrics and checking if it aligns with the business goals. About this, he said: “I think it affects the quality or what you're doing with that content too. You could do something that's salesy and really push hard to convert people, but it may turn people away or push them to sign up for GitHub stars and forget that your community isn't growing”
Conclusion
In this blog, we have explored how to measure the success of your open source community. We have seen that community is a key factor for the growth and impact of open source software, and that metrics can help you track and evaluate your community’s health and value. We have heard from experts in the field who shared their insights and experiences on how they use metrics to build and maintain strong and engaged communities around their open source projects.
We hope that this blog has given you some useful tips and ideas on how to measure your own open source community’s success. Remember that metrics are not an end in themselves, but a means to an end. They are tools that can help you understand your community better, identify its strengths and weaknesses, and improve its performance and impact. However, metrics are not everything. They cannot capture the human and social aspects of community, such as trust, loyalty, passion, and collaboration. These are the intangible qualities that make open source communities unique and powerful. Therefore, while you use metrics to measure your community’s success, don’t forget to also celebrate your community’s spirit.
By clicking “Accept all”, you agree to the storing of cookies on your device to enhance site navigation, analyze site usage, and assist in our marketing efforts. View our Cookie Policy for more information.
.png)
.png)
.png)
.png)
.png)
.png)
.png)
.png)
.avif)
.avif)
.avif)
.avif)
.avif)
.avif)
.avif)
.avif)
.avif)
.avif)
.avif)
.avif)
.avif)
.avif)
.avif)
.avif)
.avif)
.avif)
.avif)
.avif)
.avif)
.avif)
.avif)
.avif)
.avif)
.avif)
.avif)
.avif)
.avif)
.avif)
.avif)
.avif)
.avif)
.avif)
.svg)
.svg)
.svg)
.svg)
