Hello Scarf followers! I'm Shea, Scarf's new VP of Engineering. This post is to share some of who I am, what problems I'm interested in, and why that brought me to Scarf. I hope it gives you a sense of what this new role means for our mission to give open source maintainers observability into the distribution and usage of their projects!

In my career so far, I've been exposed to a wide array of problem domains and industries. I've worked in embedded hardware, industrial sensors, online retail, personalized marketing, inventory management, SaaS platforms, statistical analysis for research... It's been a fascinating journey! While my roles and responsibilities have varied, over time I've developed a number of themes in my approach to work:

• I am passionate about the big-picture product perspective. I am not satisfied writing a single line of code until I understand what value we're trying to create, for whom, and why the task at hand helps achieve that.
• I consider proper organizational structure and practices to be crucial to my own effectiveness, not to mention the overall success of the effort. Team values, formal process, responsibilities, expectations, etc. should be as carefully designed and cultivated as the product itself.
• I gravitate toward "force multiplier" aspects of the work. I'm much more likely to build a tool that helps build the end-user features than to build the feature itself, more likely to tie all of the components together than to build the component itself. My success is often measured by the problems that don't arise and the lack of difficulty others face in their work. I tend to sell shovels, not mine gold.

Essentially, I'm focused on designing and building the foundations and framework required to successfully achieve the right goals.

Barriers to composition

One of the most important cognitive methods in an engineer’s repertoire is compositional thinking, i.e. thinking about a system in terms of fundamental components viewed as units to be combined in order to achieve the relevant functionality. We can concretize this as a set of questions to ask yourself: What are the basic building blocks of this domain, what are their individual natures, and how should we put them together to build a system whose combined characteristics meet our needs?

Suppose, for example, we want to improve the development experience for our engineering team. We break down each developer’s workflow into steps, understand the importance of these individual steps to the developer’s job, and then look at how they lead into and support each other to guide our approach to implementing each individual stage and their interfaces. Or, we want to compute how much inventory to send from our distribution centers to our stores to ensure customers find what they’re looking for with minimal operational costs. In this case, we break down the computation into individual calculations that can be solved separately, identify the inputs required for those calculations (including previous calculations!), and combine all of the steps to get the desired result.

It's hardly a novel observation that composition is a central activity in software engineering (or really in any field). Why, then, is it usually much more difficult than it needs to be? If I've already done the hard work of identifying which tools would be useful for developers on my team, why do we then each need to manually fight with our package managers to get the right versions installed? I’ve figured out what each of our microservices should do and how they should interoperate. Now not only do I need an infrastructure expert to get them deployed into production, but I need a completely different system for my teammates to run them locally. Why?

In most domains, a significant amount of our time is spent translating from our carefully designed notions of our systems to the ad hoc, operational, leaky abstractions that implement them, or working around mismatches between what we wanted and what the system is able to give. And that's before we even consider composing across domains! We can do better.

Breaking the barriers with Scarf

So where does Scarf fit in? Scarf is dedicated to the same things I am: laying the social and technical foundations and building the tools to enable developers to work more effectively, to acquire better information about the value of their products, and to trade the higher quality work that results for the resources needed to go further. In particular, effective composition is a core technical requirement for Scarf.  Our entire approach is centered around understanding and improving how end-users leverage our maintainers' products to build their own, a deeply compositional process!

Scarf wants to improve composition, and so do I; I know how to do it, and Scarf can make it a success. We see an opportunity to work together to make this a reality.

And so that's what I'll be working on! There is a lot of development to do and many minds to convince; watch this space to follow along with what's next!