pmuellr is Patrick Mueller, Senior Node Engineer at NodeSource.

other pmuellr thangs: home page, twitter, flickr, github

Friday, May 18, 2007

That Darned Cat! - 2

Some more thoughts on Twitter performance, as a followup to "That Darned Cat! - 1".

Twitter supports three different communication mediums:

  • SMS text messaging
  • A handful of IM (Instant Messaging) services
  • HTTP - which can be further subdivided into the web page access, the Twitter API, and RSS and Atom feeds

I'm not going to talk about the first two, since I'm not familiar with the technical details of how they work. Other than to notice that I don't see how Twitter can be generating any direct revenue off of HTTP (no ads on the web pages, even), whereas they could certainly be generating revenue of off the SMS traffic they drive to whoever hosts their SMS service. IM? Dunno.

It would appear, or at least I guess, that most of the folks I follow on Twitter are using HTTP, rather than the other communication mediums. Maybe I'm in a microcosm here, but I'm guessing there are a lot of people who only use the HTTP medium. And there's no money to be made there.

So, we got a web site getting absolutely pounded, that's generating no direct revenue for the traffic it's handling. And it's become a bottleneck. What might we do?

Distribute the load.

Here's a thought on how this might work. Instead of people posting messages to Twitter, have them post to their own site, just like a blog. HTTP-based Twitter clients could then feed off of the personal sites, instead of going through the bottleneck.

This sounds suspiciously like blogging, no? Well, it is a lot like blogging. Twitter itself is a lot like blogging to begin with. Only the posts have to be at most 140 bytes. So let's start thinking about it in that light, and see what tools and techniques we can bring from that world.

For instance, my Twitter 'friends' are nothing but a feed aggregator like Planet Planet or Venus. Only the software to do this would be a lot easier.

Josh notes: Hmm, but doesn't RSS/Atom already give us everything we need for a twitter protocol minus the SMS (and text limit)? Indeed. Twitter already does support both RSS and Atom (I didn't see explicit Atom links, but find an RSS link @ Twitter, and replace the .rss URL suffix with .atom). They aren't things of beauty, but it something to start with. While you can already using blogging tools to follow Twitter, I'm not sure that makes sense for most people. However, reusing the data formats probably makes a lot of sense.

So, why would Twitter ever want to do something like this? I already mentioned they don't seem to be making any direct revenue off the HTTP traffic, so off-loading some of that is simply going to lower their network bill. They could concentrate instead, in providing some kind of value, such as contact management and discovery. Index the TwitterSphere, instead of owning and bottlenecking it. And of course continue to handle SMS and IM traffic, if that happens to bring in some cash.

In the end, I'm not sure any one company can completely 'own' a protocol like this forever. Either they simply won't be able to afford to (expense at running it, combined with a lack of revenue), or something better will come along to replace it.

If you love something, set it free.

There are other ideas. In "Twitter Premium?", Dave Winer suggests building Twitter "peers". This sounds like distributing Twitter from one central site, to a small number of sites. I don't think that's good enough. Things will scale better with millions of sites.

No comments: