The Cleanup

We made a big change to our site’s UI last week. Now that the dust has settled, I’d like to take some time to describe the “what” and the “why”‘s behind this change.

A little background

Although we launched 1.5 years ago, our codebase is almost 2 years old. When we began this effort, we made a bet on using Ruby, which we chose because of what I perceived as its power & elegance. Two years passed and my opinion hasn’t changed – it’s easily the most enjoyable and rewarding language to program in that I’ve had the pleasure to use.

Challenges

A few months ago, however, I started noticing that our pace of innovation had slowed down. Adding features and modifying existing ones was getting harder and taking longer. What was going on here? After some introspection, I found that it boiled down to 2 things:

Web UI

Writing features means writing new UI. With minor work, it might just be a little html. Other times, it might mean firing up photoshop, or getting our hands dirty in javascript and css. Now, as it may surprise no-one, we don’t have a dedicated web designer on our site (shocking to no-one, I suspect ;-). As a small outfit, all us developers wear multiple hats. On a practical note, this means every minute spent on UI is a minute we don’t spend on improving our back-end scalability or analysis depth. Figuring out a way to lighten our load here would help us out greatly. Ergo, our first challenge: how to reduce the amount of time we spend on our interface.

Code Size & Rot

Secondly, the code rot. The good news is that our Ruby development skills have improved significantly. The bad news our earlier-written code was now ugly (by our new-and-improved standards) – and sometimes tricky to edit and extend.

My favorite aspect of Ruby is its conciseness. As our Ruby chops improved, we found that our earlier code was “smelly”. By now we’ve gotten quite good at identifying the symptoms. Functions with vague names (and often vague purposes). Functions with more than a dozen lines of code. Duplicated code. These are the easier signs to identify.

By backporting our new-and-improved coding patterns to our earlier code we were hoping to shrink our codebase down significantly. I’ve become a big fan of small LOCs (lines of code). After all, there are only so many lines of code a developer can maintain (yet alone extend).

Solutions

In early January we took the plunge. To help out our UI challenges, we decided to make some pretty big changes:

Web UI Solutions

  • Grid-Based Css: Our prior UI was mostly developed in an ad-hoc manner. Instead, we developed our own web CSS framework with much inspiration from Blueprint. Using a grid approach reduced a great amount of guess-work involved in how to layout our pages. Not only did it reduce the amount of custom CSS required for every page, it also allowed us to spend less time inventing layouts.

  • Sparse, Whitespace Friendly Layout: We’ve removed a lot of our web-2.0 styling elements. No more rounded corners and a lot less 3d-ish panels. We’ve received some negative reactions to this (see this thread). However, by reducing the visual clutter, we are enabling our pages to become denser with data (rather than styling) AND we’re reducing the amount of time we spend on UI. These gains have not been fully realized because we have not redesigned many pages of our UI (yet).

  • JQuery: Rails has a very “close” relationship with a javascript library called Prototype. It’s really cool and we’ve had very few complaints with it. There’s been a proliferation of Javascript Frameworks, to the point where they have practically become commodities. However, I was smitten with the genius behind JQuery’s chaining model. We decided to port our Javascript from Prototype to JQuery as a result. We saw a reduction of LOC’s, but not as much as I was expecting. However, we also benefitted from simply being better Javascript developers.

  • Haml && Sass: We’ve changed how we write our html markup from the vanilla Rails-based approach (called Erb) to instead use Haml. It has radically shortened our view files and made them much easier to read and modify. Similarly, we’ve ditched our hand-edited css files and instead adopted Sass (comes with Haml). Again, our total number of css-equivalent lines of code (now written in Sass instead) – down significantly.

Code Rot Solutions

  • Rails Plugin-ization: We took many of our common patterns and created simple plugins out of them. This cleaned up our codebase and reduced the amount of unit/functional tests required. No magic here, just common sense.

  • MakeResourceful: Score another one for Hampton Catlin, the genius behind Haml and Sass. MakeResourceful is a relatively simple plugin that seems completely obvious after you try it out. It works by abstracting all the common code from your RESTful controllers, while still giving you enough hooks to modify any aspect of the behavior you want. In plain english, it shrinks your Controller code drastically. Yes – we saw amazing reduction of our LOCs. More importantly, badly-written controllers now stick out like a sore thumb. We find ourselves opening up a Controller file and either seeing very little code, or saying “hey, why is this file over 20 lines of code? Aw man – we should clean this up!”. This plugin belongs in Rails core. Kudos to nex3 one of the main devs on this project.

  • On the fence: RSpec. It might be coincidence, but it’s interesting to notice that while RSpec is really cool, one effect it doesn’t have is to shrink your LOCs. It’s a testing framework that mostly replaces the existing Rails-centric test framework. We are using this on a different codebase we’ll be releasing soon, but I have to admit that even after embracing mock-based, behaviour-driven testing, I’m still a little skeptical. We’ve written a lot of BDD tests – and they run very quickly, which is awesome. But even after all this discipline, I’m never sure the app really worked – i find myself thinking that I’m just testing that my mocks are being called in the way I expect them too. For now, at least on Ohloh, we’ll stick with good ol’ Test::Unit.

Conclusion

My first conclusion is that I shouldn’t write long posts like this. It tires me out.

Beyond this, however, is that I keep sounding like broken record when I say that there is simply a ton of cool, innovative work going on in the Open Source World. At the beginning of Ohloh, I felt a little lost trying to deal with the chaos of the open source world. Then I learned to effectively “deal with it”. I’ve since transcended into the “loving & thriving in it” camp. Where I perceived chaos, I now see strength. Diversity and competition of ideas and technologies are are healthy symptoms of adaptability and progress. The species appear and evolve, and yes, sometimes die off. But the ecosystem as a whole continues to grow and solidify.

Lastly, we’re back to moving Ohloh features forward. We will be turning our attention to our back-end for a while and see how far we can push our scalability and performance. We’d like to see how close to we can get to real-time metrics (for the record, I consider hourly-updates to be real-time enough).

Thanks for everyone’s help and feedback.

  • omatan (omatan)

    Thanks for taking the time to write this. Interesting read.

  • http://vinnl.nl Vinnl (Vinnl)

    What? Ohloh was not designed by “real” designers? Wow, I wish I could make designs like this…

    Anyway, I agree with omatan, this was an interesting read :)

  • http://phil.freehackers.org Bluebird75 (bluebird75)

    I don’t like the UI too much but now that you give good technical reasons, I feel a lot more comfortable with it.

    Can’t wait for the new features that the new UI will allow you to develop quicker :-)

  • http://mika76.wordpress.com mika76 (Mladen Mihajlovic)

    I gotta admit I don’t like the UI either. I wish you guys would just get a designer already ;)

  • jnareb (Jakub Narębski)

    I like reduction of web2.0 clutter, but what I don’t like is that borders between differents parts / boxes vanished, and it is much harder to gess which part given element belongs too: “too much air”.

  • jason (Jason Allen)

    Jakub: I agree – there are definitely some sections that need fine tuning. I’m sure we’ll get to it over time, but if anyone’s interested in speeding this up, here’s how:

    If you (or anyone else) cares to submit a screenshot w/ overlayed suggestions (like, “more space here”, or “less space here”), I promise I’ll either make the suggested fix or respond as to why I disagree.

    Thanks to everyone for the frank feedback – we’re listening.

  • http://vadi.myopenid.com vadi (Vadim Peretokin)

    I think this page:

    http://www.ohloh.net/wiki/noavailablerepository

    Looks a bit too mean, as Ubuntu doesn’t really have one. Instead it has a collection of all the packages (https://code.launchpad.net/ubuntu/+allpackages).

  • chiggsy (chiggsy)

    Right. I love the design. Function is beauty. I never saw the old design, but damn it , I don’t want to. this is perfect.

  • http://dirkjan.ochtman.nl/ djc (Dirkjan Ochtman)

    So are you getting into reading Mercurial repositories now? Would be about time…

  • moebius (Hagen Möbius)

    I liked the old design. I’m not yet comfortable with the new one. But, as stated before, you’ve given good reasons why this change is for the better and those reasons I, as a developer, can understand. I tried to design myself, I got bashed and now I delegate every tiny bit of UI design to somebody who is able and educated to do a great job with it. And on the fly I learned about what UI designers are aiming for.

    To be honest: This is what I would expect of you too. :)

    I’d like to reiterate what Jakub said: There is much more guesswork involved when trying to figure out the individual parts of a certain page. I think those pages would be much more readable if the different layout sections were highlighted with background colors which make it easier to visually differentiate them and subconciously associate colors with function blocks.

    Colors are important! Grey just doesn’t cut it. Apart from helping the viewer to grok the interface it gives your site a “face”. Currently, in my mind, the name ohloh doesn’t trigger an association or emotion, because it’s just grey.

    And there is nothing easier than looking up “color schemes” in the search engine of your choice.

    Just my 5 cent. :)

    Anyway, I’m happy that there is development, so keep up the good work!

    Hagen.