Links

pmuellr is Patrick Mueller

other pmuellr thangs: home page, twitter, flickr, github

Friday, January 11, 2008

why couchdb

Someone asked me today what I thought value proposition was behind CouchDb. Here goes.

  • It's a non-relational database. I've ranted on this before. Not like relational databases don't have have their place, but the fact of the matter is that so many people are familiar with them today, that they've become the first tool many people look at when trying to solve a persistance problem. I'm always happy to see conceptually simpler database tools get some attention, because not every problem needs a relational database.

  • It's a schema-free database. Now, you've heard me rant about how important I think schema is for things like JSON and REST, so now you're no doubt confused about why I would claim a schema-challenged database is good. The problem with schema in most relational databases is that it's terribly brittle. It's painful to deal with. Not terribly portable. Schema has a place in the world, but at the lowest level of the database, it seems like it just gets in the way. At the very least, I think it'll be interesting to see how much trouble the lack of schema for this semi-structured database causes.

  • It's a web server. W00t! My database driver is my HTTP client. Almost every language has a decent HTTP client library baked in. Almost.

  • Erlang with it actor-ish programming model. Getting some traction right on the eve of our multi-core future. The nice thing about this particular usage of Erlang is that I think it's in the exactly perfect spot. The engine. Which you never see. You'll see other languages (JavaScript, et al), but in general the Erlang is all under the covers.

  • JSON. I need not say more.

  • A 'stored procedure'-ish language, that's actually a real language. A simple and powerful language. JavaScript. And other languages, if you wish.

  • Designed for distributed usage.

  • Seems like classic disruptive innovation.

  • It's a web server and a database. And to me, this is the most interesting point. Just as we've seen client programs start to embed web browsing technology (like iTunes), there's really no reason why server programs like a database shouldn't be able to embed a web server. It's obviously a convenience factor (single install). It also has some potential nice performance aspects as the path-length between the server and the database is greatly, greatly shortened.

Let's noodle on that last point a bit more; it's a server AND a database. Now, whenever I see "server", I immediately ask the question - can I write a web app for that thing? Or is it just serving read-only documents up? Check the REST APIs. Read/write web, folks. But, there are some caveats. JSON objects are trivial to store in the database behind a simple URI, but anything else (something that's not JSON), needs to be stored as an 'attachment' for some JSON object. The raw data of the attachment is available at a particular fugly-ish URI, of the form:

/database/attachment_doc?attachment=foo.txt 

Fugly, but possible.

What does all this mean? Well, it means you could, say, serve up HTML out of your CouchDb database, that could be rendered in a browser. Which might include some JavaScript (inline, or referenced as another 'attachment'), which could then read/write objects in the database. A self-contained app inside an app server. Oh yeah, the data for the app is stored there to.

That's the theory anyway. I intend to practice this weekend.

BTW, for any Mac folk that want to install CouchDb to play, I recommend installing it from MacPorts via:

sudo port install icu erlang couchdb
sudo cp /opt/local/Library/LaunchDaemons/org.couchdb.couchdb.plist  /Library/LaunchDaemons/
sudo launchctl load -w /Library/LaunchDaemons/org.couchdb.couchdb.plist 

The first command installs the icu package, then erlang, then couchdb. The last two commands launch CouchDb, and arrange to have it started at every boot. From memory, I think I had to do these after the port install, but I could be wrong; perhaps the port install did them for you. In any case, there's no harm in running those commands again.

As a quick check to make sure CouchDb is running, either open your browser on the following url: http://localhost:5984/_utils/index.html, or run the command, which should provide the short JSON object below:

$ curl localhost:5984
{"couchdb": "Welcome", "version": "0.7.2"}

Wednesday, January 09, 2008

dear lazy web - travelling to Ireland

Dear Lazy Web,

We're heading to Ireland for a week+ vacation in March. Any must-do ideas? I mean besides testing Jon Udell's theory on Guinness. Actually, I've been to Dublin before, I can vouch for his theory already.

Here's the general plan. Fly into Shannon. Leave from Dublin. Presumably travel along the coast (southern). Though there is already some interest in going north from Shannon for the first day or two.

This particular trip we're making with two of our friends from college days, and without the kids. We've travelled with these particular friends several times over the decades. They travel more than we do, and have volunteered to get an itinerary together, along with handling reservations at B+B's and what-not, as they have for other trips we've made with them. Which is great for me. Well, easy. They've done an outstanding job of finding places for us to stay over the years. Except for "Barnacle Bill's" in Topsail Beach. We shall not speak of that.

But that means I don't have a lot of pressure to do any research. Really, no pressure at all. And to cut down on even that small amount of research that I should do, I figured I'd see if you can help. If there's something that you just know we should do, which I can propose, via ... you, please let me know. ASAP.

thx, bye

Tuesday, January 08, 2008

on tuba

Scala? Io? Factor? CouchDb? You thought those were hot memes? Fuggitaboutit. I got one word fer ya.

Tuba

Old Joe "Prognosticator" Gregorio is out in front of this one. Some very insightful thought-experiment work there. I can't wait to see how this nascent meme blooms!

Just to add some constructive information to our new tuba overlord meme, thought I'd mention my son's tuba instructor, Tony Granados, aka "Tony Tuba". If you're looking for some tuba instruction in Triangle area, we've been quite happy with Tony, as has my tuba-tooting son. Of course, Tony is profiled on TubaNews.com.

Thursday, January 03, 2008

hot data

Damien Katz, of CouchDB fame, announced today he's accepted a job with IBM, to work full time on CouchDB. For a JSON-lovin', relational-hatin' person like myself, you can imagine my reaction. "Yee Haw!" indeed!!

But when I thought about it today, I realized there's plenty of other non-relational, and sometimes web-related database stuff going on in IBM:

  • Project Zero's ZRM (Zero Resource Model) provides a REST-friendly DB abstraction layer for use in applications built with Project Zero. Though it's built for relational databases, it's one of those new-fangled ORMish things which gives you an API which is bit more 'functional'.

  • Brandon Smith (who's developing ZRM) pointed me at the Impliance research project a month or so ago.

  • Following up on the Impliance work ended up leading me to jaql.org; notice some of the same names from the Impliance paper. And the JSON; the lovely, lovely, JSON. My heart's a-twitter! Not sure if I can take the new query language; we at least need a structured query facility as well; ie, I'll give you a query expressed as a JSON structure. Early days ...

  • Can't pass up a chance to mention Jazz. The thing that attracted me to Jazz when I went to work on it many moons ago, wasn't the actual 'vertical' market they've carved out for themselves (collaborative development). It was the server. Basically, it's a versionable object database with an HTTP (but not terribly RESTy, yet) front-end. I like to think that it can easily have a life above and beyond mere "collaborative development" tools, if desired. Bill Higgins made a comment on a blog (from, of all people, Brandon Smith :-) , spilling some of those beans.

That's just the stuff I know about. And note only two of those items are actually being developed in the Information Management (IM) tower in IBM (home of DB2); WebSphere is the home of Project Zero, and Rational is the home of Jazz.

So here's a question - why do the database guys get to do all the wild and wacky stuff? Where's the wild and wacky stuff in web-serving frameworks, web-client frameworks, development tools, etc. I'll give programming languages themselves a break here, what with recent interest in Erlang, Scala, and odd-balls like Processing, Factor (ran into both of those again, over the holidays), etc. Comparatively speaking, I feel like we have a wide variety of choices of DBs, ORMs, and languages to use these days, but the other tools in the programmer's toolbox haven't evolved as quickly. AppJet, though very minimalistic, is one of the more interesting "development environments" that I've seen in a while. It's really not hard imagining what that could be on steroids. I'll just let you imagine what my Smalltalk-related comment might be here ...

Next question: when are the Information Management guys going to start working on web servers and development tooling?

BTW, I think I'll pass on Factor; I wrote PostScript programs, back in the day. Been there, done that, got the T-shirt. Though I am tempted ...

Thursday, December 13, 2007

thoughts on appjet

AppJet. Have you heard of it?

There's plenty of rocks that can be thrown here.

  • All my code needs to go in ONE file?
  • No static resources?
  • Seemingly overly-general db approach, probably yielding terrible performance
  • Where's my debugger?

But there's a whole lot more to like than dislike with AppJet. Like most of the new-fangled things on the web, I enjoy looking and learning, as a breath of fresh air in spaces that we've become too comfortable with; spaces where we've not done enough thinking outside the box (hello, relational databases!). That said, here are some thoughts.

I've been kidding my Rational brethren for years that Eclipse is dead *, everything's happening on the web, we'll using web-based IDEs in the near future. A chuckle generator for sure. Time's up though, folks. The future is here. Or at least getting very, very close.

There's lots of challenges in this space, you can see even from just a bit of playing around with AppJet. The code editing is flakey. (Peruse it with FireBug; I'm amazed it works as well as it does, honestly, given the implementation as rendering of HTML DOM; it's amazing, actually). I need to be able to edit more than one resource - I certainly don't want to have to have all my code lumped in one file; actually, I don't care if it's one file in the end, but I don't want to see it that way in the IDE. And clearly we need to be able to edit other things like HTML bits, CSS bits, etc.

It's pretty clear to me that Eclipse is too much, and AppJet is not enough, in terms of IDE capabilities, for a space like this. Whadya think - is it easier to remove function from Eclipse, or add function to AppJet? And I worry that Eclipse is veering off on a tangent (runtimes) when there is clearly plenty of work to be done in the IDE space; maybe not in the Java-hosted IDE space - what more does Eclipse really need? But what about spreading the love a little?

The code editing itself was flakey enough that I ran this on Safari just so I could see how well "Edit in TextMate" would work. It did work, but then I have two separate applications open for my "IDE", and that quickly got out of sync. Here's what I'm wondering. If these new-fangled client-based VMs (Silverlight, Flex) are really up to snuff, it should be possible to build a new text editor component with them. From scratch, painting glyphs on the canvas with low-level graphics calls. How do you think Eclipse's rich text editors work, anyway?

Then there's the database. From 50K feet, there's some similarity to couchdb here. The programming language is JavaScript, the objects stored are constrained JavaScript objects, etc. Lots of dissimilarities as well; with couchdb you have a single 'document store' with associated views; with AppJet you have multiple StorageContainers. With couchdb, an alternate 'view' of the database (filters, keyed differently, etc) are available transiently and persistently; with AppJet, the views are always transient (a classic time vs. space trade-off. Hint: storage is sometimes cheaper than time).

Just in terms of raw db functionality, what's nice is that there's not much there; just a set of simple operations you can perform against the db. You have to imagine the performance is going to be terrible on this, especially the filter/view operations. But it's clearly an interesting angle to take in the db space, and one I welcome. SQL has always given me the cold pricklies. I hope that the work that's gone on in recent years introducing XML (another thing you happily won't find in AppJet) into the database world, might have paved the way for JSON and JavaScript. Are you listening, Anant?

This space, in general, has lots of low fruit to be picked; it's great to see things like AppJet making a run for it. I certainly hope that someone doesn't buy them and then send them to a dark place.

For more info, Dion Almaer posted a short interview with an AppJet dev yesterday.


* update on 2007/12/13 at 1:00pm

Of course, no direct offense to Eclipse meant here; I use it on a daily basis, and simply couldn't live without it. I meant to implicate the whole notion of desktop-based development, Eclipse being my flavorite for Java, in light of our new web-based IDE overlord takeover; or at least the possibility of that happening. I'll try to attack things in a more broad sense in the future. :-)

Thursday, November 15, 2007

frogpad

In "iPhone and text entry" K1v1n complains about the iPhone / iTouch 'keyboard', and wonders if instead of spending time correcting the endemic finger checks, "Can we libe with good eoifg?"

No, we can't.

Here is a step in the future direction we'll be going with mobile keyboards: Frogpad. Obviously not good enough; it's too big. I'm thinking I'd want something like the stick of a joystick, that I could grip and do entry with my fingers, with buttons where the fingers rest. TrackPoint on the end controllable via your thumb, for moving the 'cursor'. Hell, I'll live with one finger button and learn morse code.

Anything is better than a soft keyboard. Multi-tap is better than a soft keyboard.

The funniest thing to me is that the assumption that a soft keyboard is deemed acceptable at all. We've been stuck with this same damn keyboard layout for over 100 years. I think we can move on. Apple, you're supposed to be innovative, right? Let's see some real innovation, please.

Once we have that nasty 'keyboard' problem solved, and since we already have the audio problem solved, the last problem is video. That solved, I can just keep my iBoxen, which only needs an on/off switch and maybe a power plug, in my pocket or man-purse, where it belongs.

We're not talking about flying cars here folks.

BTW, the FrogPad looks like it would be fun to try as a MIDI controller, except for the fact that it costs $150.

Tuesday, November 13, 2007

initial android thoughts

android wallpaper thumbnail

Some initial, random thoughts / wild speculation on Android, given the limited amount of information currently available.

  • I don't think the mobile app 'problem' has ever been about the programming experience. Frankly, all the wonky tools, languages, frameworks are nothing but an enticement for interested hackers. More puzzles. Certainly, that's why I got involved with these guys about ten years ago. I'll go ahead and make the obligatory Smalltalk reference now, and then just shut up about it, mkay?

  • No, the problem with the mobile app 'problem' has always been about lock-in by the various entities. Service providers that charge exhorbitant prices for network data access; that don't allow the download and install of Joe Rando application, instead, monetizing the application story; not even having a toolkit available to build apps in the first place. Maybe times have changed since I stopped paying attention to the market, but I doubt it. Perhaps Google and indirectly Apple can help break down some of these barriers. That's the real problem to be solved. Or perhaps Google could more directly fix the problem. I should be able to write an app for my mom that she can easily install on her plain old phone. That's got to be the long term story, anyway.

  • Android is Java, and not Java. You write your application in Java, but it's translated to Dalvik format for running on the Dalvik VM. The Dalvik VM is not a Java VM, presumably.

  • Dalvik is a registered-based VM, instead of a stack-based VM. What does that mean? A friendly link in the android mailing list points to an document describing the inferno VM. Note one of the authors, Rob Pike. How conveeeeenient. :-) BTW, he's got a great email address!

  • I've heard folks claim "it's not J2ME", or whatever it's called today. Not quite right. Esmertec says it will support J2ME per customer request. Presumably, customers being service providers. But you could imagine Sprint charging you a few bucks for a MIDlet capability. Of course, your MIDlets will have to be modified to Dalvik VM format, presumably, before run.

  • There are actually three J2ME-ish packages included in the android.jar file that shows up in Eclipse when you create an "Android project". javax.microedition.[khronos.opengles|lcdui|lcdui.game], though the packages appear to be radically stripped down. Why are they even there? The khronos.opengles package appears to be 'public', not sure why the others are there.

  • Apache Harmony classes are included. Any Java lover that ever dissed Apache Harmony as pointless, needs to eat crow right now. Because, had it not been for Apache Harmony, you might be programming Android in Python instead of Java. Or C++.

  • Includes an XML pull parser. Thank &diety;! Well, it's in the android.jar used in Eclipse, but I don't see how you get to it via published API.

  • Includes SQLite. I'd rather have seen LittleTable. heh

  • The .class files in the jars seem to have (or at least some have) debug information. Decompile away! Though you're probably not allowed to per whatever click-through license you clicked through.

  • Ian Skerrett has a blog post titled "What Does Android Mean for Sun's OpenJDK" which is pretty self-descriptive. Read up, if you're interested in licensing hoo-haa, especially the Java variety.

  • Miguel de Icaza has some thoughts on "Android's VM" and Mono. We'll see a lot more of this, including folks wanting to target existing languages to the Dalvik VM, presumably. I wonder if any of the PyPy developers work at TheGoog?

  • Could 'typical' JavaDoc have been so hard to build?

  • I got to install Gutsy in a VMware session on my MacBook, and then Java, and then Eclipse, and then the Android SDK, then the Android plugin. Because there's a little problem running the Android plugin in Eclipse for some Mac users. Perhaps that's why there's soft keyboard in the emulator?

  • Personally, I think Google is shooting a bit low here; the platform sounds like it would be good enough to run little 'mini-apps' on your desktop. Like say, AIR does. Or like Google Gadgets.

  • It's not a real platform until it has IDL. Woo Hoo!!

Friday, November 09, 2007

how buildings learn

My hall-mate Bill Higgins has often recommended the book "How Buildings Learn" by Stewart Brand as "the best book about software design that never mentions software". He's right, it's a great book, and although it doesn't talk about software, there's a lot to take away from a software perspective.

In general, Brand's premise is that most buildings have fairly long lives, and end up serving multiple distinct uses during their lives. Those uses end up forcing the buildings to change, and much of the book discusses aspects of buildings that work in favor of buildings successfully changing to suite new uses, and factors that work against it.

In terms of software, I think one of the sins we most frequently commit is not future-proofing our products. If your product is successful, it might well have a very long life. Have you constructed it such that it can easily be modified to handle future requirements placed on it? The answer is typically, no. For example, see Steve Northover's "API grows like fungus!".

I don't want to spoil the book for you, but I will advise you to do this: after reading a section of the book , spend a little time imagining how what you just read applies to software development. It doesn't all apply; but a lot does.

If you've already read the book, you'll get a kick out of the following article from Slashdot this week:

MIT Sues Frank Gehry Over Buggy $300M CS Building

If you haven't read the book, here's a little spoiler; Brand spends a number of pages complaining about a different "modern architecture" building on MIT's campus, the Weisner Builder. Brand also spends a fair bit of time complaining about problems endemic to modern architecture, including leaks, and included the following quote talking about Frank Lloyd Wright:

His stock response to clients who complained of leaking roofs was, "That's how you can tell it's a roof."

Note Brand's book was published in 1994; construction on the Stata Center (Gehry's building) began in 2001. The Stata Center replaced Building 20, which was the 'vernacular' building that Brands praised so highly. Buildings might learn, but people apparently don't.

The only real complaint I have is the form factor of the book. It's 11" wide by 8.5" high (or so). The only comfortable place I've found to read it is in bed. I assume the layout was chosen because the large number of photos included in the book; it might well have been the only practical way to publish it. But it does seem a bit ironic, for a book that is pushing "function over form", to be such a difficult physical read.

Highly recommended.

defining simple

Let me expound a bit on "simple", since Sam referenced me.

  • I consider myself a Ruby n00b.

  • I'm very familiar with HTTP.

  • I'm pretty familiar with AtomPub, and the Blogger posting interface is basically AtomPub-ish.

Most of the time spent writing the feedBlogger.rb script translating the pseudo-code in my head into Ruby. Also figuring out the ClientLogin protocol. The Blogger Data API Reference Guide was all I really needed to do the business logic. Not much there, but there's not much to know, assuming you're already familiar with Atom and AtomPub.

So keep in mind, this is fairly low-level code, executing HTTP transactions pretty close to the metal, from someone familiar with doing that. And it's a one-off program, didn't worry about error checking, caching, etc. Things that a program which isn't a one-off would like have to concern itself with.

If you're lucky enough to not be working in Ruby, you might have gotten by with one of the Google Data API Client Libraries, available for Java, .NET, PHP, Python, Objective-C (???), and JavaScript. Not sure if you could really do what I did, with those libraries or not, since I couldn't use them, so I didn't even look at them.

Now, as a thought experiment, let's consider if the Blogger API was actually WS-* ish rather than RESTful.

  • In theory, I could take a WSDL description of the API, and use it with a dynamic WSDL/SOAP library, if one exists for Ruby (I know one exists for PHP).

  • Or I could generate some stubs in Ruby for that I could call as functions, assuming there is a stub-generator program available for Ruby.

  • Or knowing HTTP pretty well, as well as SOAP (not much there), as well as being able to translate WSDL into an HTTP payload because I'm familiar with THAT story, I could have written it in a low-level style as well.

The problem with the first is the first option is dealing with a thick runtime stack that I might have to debug because it's doing a lot of stuff. The typical problem is dealing with the XML payloads; getting all the data generated the way the server expects it. I've had to deal with that many, many times over the years.

The problem with the second is that there's additional stuff I have to do, and maintain, for my simple little program. Instead of just a script, I'd have to keep the WSDL, the build script to generate the stubs, and the stubs themselves. And even then I'm still left with a thick runtime library I might have to debug (see paragraph above).

The third option is just more work I'd have to do, and generally yuckifying the client program even more.

However, let me throw a little water on the REST fire. The Google Data API Client Libraries also suffer some of the same issues with WSDL. In particular, while I think people have the impression that RESTful clients can get by with just a decent HTTP library, the fact of the matter is, that's too much for some folks and having client libraries available is a nice thing to have. Which I will have to maintain with my programs.

And I did have to do some squirrelly stuff with the XML payload I was sending the Blogger; bolting the Atom xmlns onto the entries; there's probably a cleaner way to do this, so it might just be my unfamiliarity with REXML.

The big difference, to me, is that REST is less filling, as opposed to tasting great; ie, there's still a fair bit of highly technical skill needed to use this stuff, there's just a little bit less with REST compared to even simplistic WS-* WSDL/SOAP usage.

moving back

As I mentioned on my developerWorks blog, I've decided to do my blogging back here at Blogger.

As part of that process, I set up a virtual feed at feedburner, so that when I move next time, it'll be complete transparent, for feed aggregators anyway.

Moved my content over with a simple matter of programming.