Source Code Systems & Quality

Robin and I have a recurring debate regarding projects without source control systems. But before I explain the debate, a little background is required (ok, skip the next 2 paragraphs if you must).

Ohloh only offers development metrics to projects with openly-accessible source control systems. That way our analyses can go back in time and get a clear picture of how the project was built. Even with only CVS, Subversion and Git source control system support, we get pretty good coverage. However, it’s not universal.

A major hurdle are projects that don’t expose their source control system at all. They typically offer up a snapshot of their source code instead. We’ve been considering adding support for Ohloh analyses for this specific case. Of course, the analysis would be downgraded: no contributor information or timelines could be determined. However, we could still detect source code licenses, languages and overall codebase costs.

This is where the debate comes in: Robin argues that projects that don’t make their source code repositories public are highly suspect and Ohloh shouldn’t bother trying to provide degraded analyses. In other words, if you’re considering using such a project, the sheer fact that they don’t make their source code control system public should be enough of a warning for you to reconsider.

My position is slightly more conciliatory. I agree in spirit with Robin but realize that there are probably too many special cases for us to generalize this way. A good counter-example is PCRE. It’s written by a single developer who doesn’t bother using source control (why? see below). However, it’s a well written, very useful library. Right now the Ohloh project page for PCRE is very empty. Ohloh can’t connect to his repository — he doesn’t have one, remember?

Shouldn’t we at least provide some basic info instead of nothing?

Well, here’s Robin’s summarized position: there are 3 possible reasons projects don’t expose source control systems:

  1. they don’t use one
  2. they are too lazy to expose it
  3. they have something to hide

When it comes to picking an open source app/library, none of these reasons instill much confidence. So in the end I have admit: I agree with Robin.

Given our limited development resources, we’ll skip supporting non-repository projects for now.