Building Scarf: Avi Press on Haskell, Telemetry, and Open Source Challenges

In this episode of the Haskell Interlude Podcast, Joachim Breitner and Andreas Löh sit down with Avi Press, the founder of Scarf, to discuss his journey with Haskell, the telemetry landscape in open source software, and the technical as well as operational challenges of building a startup with Haskell at its core. As a company dedicated to providing analytics for open source software, we use Haskell to build our product. The reason is simple. As Avi Press puts it, “Haskell is a fun language to work with. The community is great. […] And the ecosystem is lovely.”

Getting into Haskell

The conversation begins with Avi recounting how he got into Haskell. Avi’s programming background is rooted in his education at the University of California, Berkeley, where he acquired skills in Lisp. Despite working with languages like Python, Scala, and Java, Avi’s affinity for pure and lazy paradigms led him to ultimately explore Haskell seriously around 2014. After a few uneasy starting chapters, Avi found his stride and experienced what he describes as “a truly magical programming experience.” He then ventured into the open-source software ecosystem, contributing to several functional libraries. 

Scarf’s Story

Avi’s passion for Haskell carried over when he founded Scarf. Surrounded by the ups and downs of open source maintenance, he and his team sought to leverage Haskell to build a platform for deeper and more actionable usage analytics. With immense learning curves and initial setbacks such as community backlash over telemetry practices, Scarf evolved by listening to user feedback and adapting its methods. 

Scarf seeks to empower open source maintainers, contributors and commercial projects with actionable insights into user behavior, allowing them to make data-driven decisions. Scarf aims to help maintainers optimize their workflows, reduce burnout, and focus on tasks that yield the greatest impact. Our mission is to turn intuition into informed strategy, ensuring the development and evolution of open source projects are guided by real user data.

The first commit in Scarf’s repository was done in 2019. Five years later, Avi’s development team of 16 people is developing the code base with 350k lines of Haskell. 

Challenges and Triumphs with Haskell

Avi gives a candid overview of both the joys and hurdles of using Haskell. Haskell lets you change large parts of the system quickly and confidently, which is great for early-stage development. Its speed makes prototyping easy and fun, keeping developers engaged. 

However, Haskell also presents significant challenges—chief among them being the small ecosystem, outdated libraries, and the complexity inherent in interfacing different Haskell ecosystems. Avi notes that many times, his team had to write their own bindings for databases and manage frequent breakages due to updates in libraries and GHC. This encouraged the Scarf team to dedicate time to OSS, partially bridging the gap in the ecosystem. While the benefits are promising, the drawbacks remain significant.  

Hiring and the Haskell Community

Hiring Haskell developers is one of the nuanced challenges Scarf faces. Avi underscores the difficulty of finding experienced Haskellers in specific locations, thereby making Scarf a remote-first company. While initially, they had an influx of talent mainly due to the novelty and charm of Haskell, scaling this talent pool presents its own set of problems.

Moreover, Avi advocates for a more welcoming and larger Haskell community. By streamlining the entry points and smoothing the learning curve, Haskell could attract more developers, bringing much-needed diversity and growth to the space.

Performance and Ecosystem Needs

Haskell itself provides excellent performance, as Avi notes with Scarf’s own robust, high-performing gateway services. Yet, operational challenges like managing space leaks and the occasional scarcity of runtime troubleshooting tools remain a hurdle.

He also points out that Haskell’s performance and type safety come with trade-offs at the beginning of a project lifecycle, where rapid iteration and adaptability might sideline the need for the language’s strong correctness guarantees.

Vision for the Future

Avi shares his vision for the Haskell ecosystem:

- Improved Onboarding: Making the learning process smoother and more engaging to reduce the drop-off rate from new learners.

- Ecosystem Growth: A concerted effort to stabilize and expand the library ecosystem, reducing the need for custom solutions and fostering broader industrial application.

- Industry Participation: Encouraging more industrial users to contribute back to the community through open sourcing tools and libraries.

- Enhanced Tooling: Addressing developer experience paper cuts, ranging from better error messages to more robust developer tools.

With candid reflections and forward-looking aspirations, Avi’s insights serve as both a testament to Haskell’s strength and a call to action for its community. The shared journey of Scarf hints at the potential Haskell holds, not just as an academic curiosity, but as a cornerstone of agile, innovative startups.

The dialogue closes with a note of optimism—acknowledging both the current strides and the road ahead for Haskell. Avi’s experience underscores that while Haskell’s steep learning curve and ecosystem challenges are real, they are balanced by the language’s unmatched strengths and the vibrant community supporting it.

Avi believes that by aligning around these shared goals, the Haskell community can transition from niche brilliance to a powerhouse of practical, industrial-level significance.