Ollama Ahead of Azure? A New Leaderboard for Open Source AI Adoption

Scarf sees billions of open source (OSS) package downloads each day, and an increasing number of these packages are AI-focused, often even tied to particular AI platforms. This makes for a unique dataset to monitor global AI adoption grow around the world.

Today, we’re releasing the OSS AI Index. It ranks AI model providers and inference platforms based on the number of distinct organizations Scarf observes downloading packages associated with them. We will keep this index updated as a source of up-to-date information on trends in OSS AI and how they are changing over time.

We hope to see the OSS community continue to push the frontier of whats possible with AI and the positive impact it can have on the world. Scarf recently co-signed the open letter for Open Weights and American AI Leadership as part of our support for a future where AI progress is in the hands of everyone and not just a select few.

What the index measures

We rank each provider by the number of distinct organizations Scarf observes downloading associated packages on a given day. We count an organization once per provider per day, even if it downloads several related packages or downloads the same package many times. This keeps the index from being skewed by any single organization skewing results with large deployments.

To keep an apples-to-apples comparison across the list, we separated this index into two list — “model providers” vs “inference platforms”. Many companies appear in both lists.

What it does not measure

Package activity alone can provide a unique and robust signal, but it’s important not to over-generalize the results, because there are notable signals that it does not capture. We do not observe the underlying model consumption, API calls, token volume, revenue, or model performance. The result is a directional view of open source package adoption with respect to these companies.

It’s important to emphasize that we avoid inferring the model behind a platform. If Scarf observes an Ollama package download, we can attribute that signal to Ollama and not what model was used with it. While there are OSS packages that are explicitly tied to a model, this is not common enough at this time in our data set to make strong claims.

How we built it

We build this index to track package downloads across every major registry Scarf tracks. That includes downloads we observe through Scarf Gateway, as well our the package registries we partner with. We combine those sources for a cross-registry view of AI package adoption.

A core piece of this work was identifying which of the nearly 700k tracked OSS packages are tied to a particular model or inference provider, which ties into existing internal work we’ve done on a unified cross-registry OSS package index.

We use that package -> provider mapping, along with our state-of-the-art enrichment pipeline to keep measure how many organizations are downloading packages tied to each provider over time. The publicly published index will start by showing how each provider has moved on the index over the last 30 days.

There are many OSS AI platforms that we are aware of but still lack sufficient data to include. Those may be added in the future as we acquire new data.

What the first snapshot shows

Some of the initial results may not be surprising. OpenAI is the default choice of many developers building with AI and has a large lead. Amazon holds two of the top four positions in the inference-platform view, between Bedrock and SageMaker.

One result that may be surprising is just how pervasive open platforms like Ollama are performing, ranking #6 in the inference-platform view as I write this, even ahead of large platforms like Azure. I see an open source project built around local model execution reaching that position as evidence of a enormous market for open source AI infrastructure. This would be easy to miss without data from OSS consumption.

Kimi stands out for a different reason that may be a surprise. While Kimi has pushed the state of the art in open-weight models, they sit relatively lower in the package-adoption rankings. This may be yet another example of a challenge many in the OSS world are all-too familiar with: releasing great OSS (or in this case, an open-weight model) does not guarantee that the developer will effectively capture the surrounding usage, market share, and/or commercial value. Again, this index does not measure revenue or model usage directly, but this index does surface differences the are downstream from each provider’s distribution strategy.

Why I think this index is interesting

If you focus on model releases, benchmarks, and product launches, you miss a large part of how developers build actually build with AI over time. With this index, we can monitor how these tools are actually proliferating and get a clearer picture into what is coming next.

People building open source AI projects can use this index when deciding which providers or runtimes to support. AI providers and researchers may find this list useful to see how they are ranking in the OSS space. The finance world is already learning how to use trends in open source to make more accurate predictions of technology trends more broadly.

Follow the index

We will be updating the Open Source AI Package Usage Index daily. We are starting with a conservative catalog and two public views. We will expand coverage as we find more packages we can associate with providers.

If you maintain an AI package we should consider or see an error in how we represent a provider, get in touch with us at help@scarf.sh. Contact us there for more granular data, a live feed, historical datasets, and more.