At its core, developing successful products is a matter of listening to users and tuning everything to their needs. ‘Listening’ can be a misleading word however, as often users don’t tell you what they need explicitly — and that’s where analytics comes in. By gathering and analyzing information from within your product you can identify areas of focus and generate ideas on how to improve.

There are a number of tools you can use to collect data about how your product is used, including open-source software such as Scarf or PostHog. In this article we’ll explain three ways you can use such tools to improve your projects.

Funnels and metrics

Funnel analysis enables you to see how a group of users move through a sequence of actions. A typical example would be a sign-up or deployment process, wherein you track how many users discover your project, sign-up and then deploy successfully. 

At PostHog, funnels are one of frequently used tools for engineers and we represent the information visually so it becomes trivial to spot areas for improvement. Some organizations, such as the open-source GraphQL engine Hasura, have even used funnels to improve their onboarding processes by as much as 10-20%. 

However, it’s also possible to apply the funnel concept to an entire project. The AARRR ‘pirate metrics’ framework is a system to track how successful a project is by monitoring the following steps:

• Acquisition (How many users find your product?)
• Activation (How many users use your product?)
• Retention (How many users continue using your product?)
• Referral (How many users tell others about your product?)
• Revenue (How many users pay for your product?)
  

Don’t be fooled by the cute name — the pirate metrics framework is an essential tool for any modern product and enables you to quickly isolate where you should be focusing your efforts.

Documentation

At PostHog, we don’t approach documentation as merely a manual we have to write; we view it as part of the product itself. We put a lot of effort into creating new product analytics tutorials and accepting feedback or updates from the community. It was this focus which even led us to discover Scarf!

Part of the reason we believe it’s important to analyze documentation usage because it’s a way in which we can get further insight into what users really want to accomplish and where they experience friction. If, for example, we see that users are spending a lot of time reading about how to use session recording then we can infer a high level of interest in that tool — but if we see the actual usage is low then we know that we’re falling short of the mark. 

Documentation is also especially important for open-source projects, where users rely heavily on READMEs and deployment tutorials. Scarf’s Documentation Insights tool is tailor made for such occasions, enabling you to measure which parts of your documentation are most popular and with whom. 

Experimentation

Once you’ve begun using the AARRR framework and tracking users across your product and documentation, it’s time to take things one step further and start testing changes. 

At PostHog, we test product changes by rolling them out behind feature flags which are enabled for a fixed percentage of users or organizations. This obviously enables us to collect qualitative feedback from the test group, but it also gives us a control group to compare quantitative results from. 

Using feature flags in this way is especially powerful because you can steadily increase the number of users in the group, or disable it instantly if needed. Other successful products, such as MentionMe and Phantom in this way, though dedicated experimentation tools such as Optimizely offer alternative methods. 

Whatever tool you use, constant experimentation is essential to making sure that changes and updates drive your project forwards, rather than back — another example of how analytics can help you build better products. 

--

Joe Martin leads Product Marketing at PostHog, the open-source product analytics suite which you can host yourself.