« April 2006 | Main | June 2006 »

May 29, 2006

Toward Actual Open Source Web Applications

In the twinkling buzzword constellation that constitutes the Web 2.0 firmament, "open source" is one of the brightest and most twinlky of stars. In Tim O'Reilly's original article announcing the concept, the phrase appears in no fewer than seven different (highly jargonic) contexts: (1) commodity servers running open source operating systems, (2) open content creation through peer-production methods, (3) open editing of collective knowledge repositories, (4) open architectures that build collective value automatically through users pursuing their own "selfish" interests, (5) users treated as a collective of co-developers, (6) software that remains in perpetual "beta" -- a state of constant development in which the product is never declared finished -- and (7) open access to data via RSS and APIs.

As O'Reilly tells the tale, contemporary web applications have interacted with the open source software community in just about every way possible: they've consumed its products, they've aped its architecture, and they've even co-opted its rhetoric and style. The one thing they haven't done is actually open source their own applications. And I'm beginning to wonder why not.

Why aren't there any websites that are truly open source? Why aren't there any websites that are actually written and maintained by a hierarchy of volunteers checking out and modifying liberally-licensed source code from a public repository?

Open source thrives in the context of commodity software that's needed by a large number of people, especially geeks, and for which branding and marketing provide little competitive advantage: think Linux, Apache, and Firefox. The web certainly has no shortage of applications that fit this bill (search and wikis come immediately to mind). So what are the practical obstacles preventing open source coders -- who've rushed eagerly into just about every other class of software -- from building web applications? What's the hold up?

One obstacle might be infrastructure. Where desktop and server software can be easily compiled and run locally on each individual contributor's computer while they work on it, mainstream web applications (e.g., Google) require extensive server hardware and voluminous bandwidth capacity far beyond what is available to the average web coder.

On the other hand, these need not be barriers to the development of such applications, merely their deployment. Contemporary development platforms, such as Ruby on Rails, come packaged with everything necessary to be built, tested, and run locally including a lightweight development server, a full-stack application framework, and a fully-integrated testing environment. With smart use of good testing practices such as mock objects and fixtures the run-of-the-mill personal computer can become a viable development platform even for large and complex web applications which require enormous quantities of data. And further, if such an application were designed in the emerging contemporary style as a series of independent modules which communicate via an API, the problem would be even further reduced, becoming mostly one of coordination and management like any other open source effort.

And the deployment problem is not necessarily intractable either. I see two potential solutions: the benevolent giant and the swarm of peons. Think of the enormous server and bandwidth resources devoted to open-in-spirit web applications by such non-profit organizations as the Wikimedia Foundation, and the Internet Archive. An entity at that scale could easily step up and provide the necessary infrastructure for an open source web application that was being written by volunteers. Further, such an organization would almost definitely have the clout and community mindshare to create such a project in the first place if they wanted.

The second potential solution is a little more out there. With just a dash of wickedly ingenious infrastructure mojo, the massive computing demands of a start-up open source web application could be distributed amongst its organizers, maintainers, and testers themselves. Using a shared computing architecture along the lines of Berkeley's SETI at home, the big chunks of data processing required by, say, the spidering and indexing capacity of our open source search engine, could be broken up into bite-sized bits and chewed up by a network of volunteer personal computers.

I'm sure each of us could think of a handful of other obstacles, both technical and social, facing anyone wanting to build a true open source web app. But I'm also sure that if the conversation got started, we'd just as quickly start coming up with solutions. That's the advantage of open source. As the old chestnut -- which I'm sure I saw quoted in O'Reilly's essay -- goes, "with enough eyes, all bugs are shallow".

Tagged: , , , , , , , ,

Posted by Greg at 3:02 PM | Comments (0)

May 18, 2006

Back from the Lull

These last forty days or so have been the longest I've ever gone without writing on this blog since it moved to UrHo. I actually managed a long enough absence that the whole template broke down due to lack of content, leaving only the header, footer, and right side bar visible. Sorry about that. And thanks to all of you who pointed that particular problem out to me. It shouldn't happen again

Even though, I've never been the most regular of bloggers, to my credit, there were some actual reasons that this gap went on as long as it did. And, hence, I have some things to talk about now that I'm back. First of all my band, At Dusk finished a new record called You Can Know Danger and I totally rebuilt our website to match the aesthetic of the new record. Look for a series of posts on the strategy I took ("How To Make Your Band Website with Web 2.0") to come soon. Plus we just got back from our first tour in support of the new record which encompassed nine shows in ten days in California. Keep an eye on the At Dusk Flickr Group for pictures from that.

Also on the music website front, before leaving for tour I totally rebuilt the PDX Pop Now! site in Rails. The new site's really exciting. It's got a super-minimal content management system that lets all of our board members add and edit content. This means that as things go forward, I'll have more time to develop new features that we've wanted for a long time like actual RSS feeds and maybe even forums, since I won't be the only one editing the live site. Also, we're now taking requests for who you'd like to see play the festival. Unlike in the past where we made Portlanders jump through complicated and often malfunctioning hoops before being allowed to put in their two cents about who they wanted to see play, this year we're keeping things super simple. As many people as want to can just show up, tell us what band they'd like to see, and bam it gets added to our running list. Totally easy. You can do it as many times as you want. We're going to let the computer do all the work this time to figure out what's actually going on. Plus, we've got an XSPF-based player through which you can listen to all of the bands that submitted their music for consideration for the compilation. I didn't realize it until I had it completely built, but it is probably the biggest online collection of Portland music that's ever been assembled. If you want to know what Portland sounds like, there's really no better place to start.

All of this combined with my actual paying programming jobs added up to no time for blogging. In the next couple of weeks, I'm going to try to more than make up for it. I've got long posts planned, instructive series, short blurbs on artists, lots of stuff. So, thanks for hanging around through the lull. It'll turn out to have been worth it. I promise.

Tagged: , , , , , ,

Posted by Greg at 11:22 AM | Comments (3)