BYBLACK DUCK

Adoption Metrics and Ohloh

By on November 7, 2012 in blog

Simon Phipps disses downloads as a proxy for FOSS adoption in his post for InfoWorld entitled: “The Download Deception”. He points out how important community metrics are in assessing FOSS projects, and gives Ohloh a nice shout-out in reference to his experience at Sun Microsystems, assessing Sun’s successes and problems with their open source communities:

If we’d had tools like Ohloh in those days, I’m sure things would have been different.

This begs the question though: if adoption is truly an important metric in understanding the success of a FOSS project, how could a site like Ohloh do a better job in illuminating adoption trends for projects? We could look at site statistics, downloads, questions on StackOverflow, and many other potential proxies for adoption, but we’re curious – what do you think? Please add some comments to this post with suggestions for new adoption metrics that Ohloh could .. adopt!

About Rich Sands

I'm the Director of Developer Communities at Black Duck Software, and the product manager for Ohloh.
View all posts by Rich Sands →
  • http://twitter.com/ZeroOne3010 ZeroOne

    I have also never quite understood the relevance of the number of downloads as an adoption metric. Say your program has 1000 users, you release a new version each month, and the program automatically updates itself. Does that count as 12000 downloads per year, then? Contrast that to a scenario where you have 5000 users but you release twice a year: that’d account to just 10000 downloads a year, even though you actually have five times the user base that the first program has.

    Off the top of my head I can think of a few other adoption metrics. First, the number of followers a project has. Many of those hosting sites, such as Bitbuckest, allows you to mark a project as favorite, so you become its follower. Since becoming a follower always requires the user to take some action, I find it would be a relevant number in tracking the interest the project has.

    Similarly, the number of forks a project has would also be a good metric, as it signifies that the one who forked the repository is also interested in the code itself, and not only the end product.

    Finally, maybe a little less obvious but still a nice metric would be the number of issues in the project’s issue tracker. How many new issues are added per month? Who adds them: everybody, or just the core developers of the project itself? Are issues also resolved? What’s the rate of new issues versus issues that get resolved? It’d be great to see some graphs on those things. I don’t think I have ever seen an issue tracker actually do that by itself. :o

    • richsands

      @ZeroOne thanks for your comments! Ohloh has long had an adoption metrics feature – the “I Use This” and stacks functionality. It only tallies data on Ohloh account holders and their use of projects however. Followers is a good one – it shows interest anyway, but not all forges have a “Followers” feature.

      I think of “forks” as more of a contributor metric – as you said, it demonstrates interest in the code. I think of adoption as being more about who is using a code base – incorporating the code, or calling the API, or using the client app, or deploying the server infrastructure, etc.

      Do you think there is a relationship between adoption and popularity? Could social media signals help here?

      Thanks again!

      • http://twitter.com/ZeroOne3010 ZeroOne

        I think forks also signify adoption — what if I fork a project, tune it a little for my own needs and then deploy it? Shouldn’t that still count towards the popularity/adoption of the original project? Also, from forks you get pull requests, and I think those at least would signify adoption. After all, why would you contribute a feature or a bug fix to a program you didn’t use by yourself? Then again, maybe that does happen, and maybe the creator of the pull request would also be a follower of the original project and would be counted in there, but anyway.

        Also, I’m certain that there’s correlation between adoption and popularity. Adoption should be the easier one to measure, popularity is more of a subjective thing: if your company has adopted a given framework, it still doesn’t mean it’s popular within the employees. I guess social media signals would help to measure the actual popularity. However, for less known products, like, say, 95% of all Google Play Store apps, the signals would be very very weak. Anyway, Google Play Store supports “+1′ing” the apps right there, and that would be one way of measuring the popularity of Android apps at least. Then there’s the Windows Marketplace and Apple’s App Store which I’m sure have similar features, but regular PC open source software doesn’t have such a centralized marketplace where you could easily measure popularity.

  • Jakub

    There has been the OpenSolaris project on Ohloh since the very early days – 2007 to be exact. So the metrics have been there all the time. All what was needed was to go and see the bloody numbers.

    • richsands

      @847a100835ca18d253dafe786d621da4:disqus You’re right – Ohloh could have provided an objective view on OpenSolaris community. Back when I was at Sun working on OpenJDK I was unaware of Ohloh. Hopefully we’re evolving how we visualize and report on these activity metrics so that they’re more easily usable by project managers. Oh – and becoming more well-known also!

      Do you have any ideas how a site such as Ohloh might better roll up adoption metrics? I’ve been thinking “adoption” means different things for a client app, a server app or infrastructure, an API library, a mobile app, etc. – so there might not be one answer. Thoughts?

  • jg114

    Badges. As developers, we might like to advertise our expertise. As project managers, we would like to know who knows and uses our software. Rather than having each project set up their own badge system, Ohloh could put in the infrastructure for all project managers to set up the criteria for awarding badges for users and developers to earn.