pmuellr is Patrick Mueller

other pmuellr thangs: home page, twitter, flickr, github

Friday, October 12, 2007

on links

When people think about links, with regard to tying together information on the web, the usual thoughts are of URLs. Either absolute URLs, or a URL relative to some base (either implicitly the URL of the resource that contains the link, or explicitly via some kind of xml:base-like annotation).

But I wrestle with this.

Here's one issue. Let's say I have multiple representations of my resources available; today you see this typically as services exposing data as either JSON or XML. If that representation includes a link to other data that can be exposed as either JSON or XML, do you express that link as some kind of "platonic URL"? Or if you are doing content-negotiation via 'file name extension' sort of processing, does your JSON link point to a .json URL, but your XML link point to a .xml URL?

See a discussion in the JSR 311 mailing list for some thoughts on this; I stole the term "platonic URL" from this discussion.

The godfather had something interesting to say in a recent presentation. In "The Rest of REST", on slide 22, Roy Fielding writes:

Hypertext does not need to be HTML on a browser
- machines can follow links when they understand the data format and relationship types

Where's the URL? Perhaps tying links to URLs is a constraint we can relax. Consider, as a complementary alternative, that just a piece of data could be considered a link.

Here's an example: let's say I have a banking system with a resource representing a person, that has a set of accounts associated with it. I might typically represent the location of the account resources as a URL. But if I happen to know, a priori, the layout of the URLs, I could just provide an account number (assuming that's the key). With the account number, and knowledge of how to construct a URL to an account given that information (and perhaps MORE information), the URL to the account can easily be constructed.

The up-side is that the server doesn't have to calculate the URL, if all they have is the account number. They just provide the account number. The notion of content-type-specific URLs goes away; there is only the account number. The resources on the server can be a bit more independent of themselves; they don't have to know where the resource actually resides, just to generate the URL.

Code-wise, on the server, this is nice. There's always some kind of translation step on the server that's pulling your URLs apart, figuring out what kind of resource you're going after, and then invoking some code to process the request. "Routing". For that code to also know how to generate URLs going back to other resources, means the code needs the reverse information.

The down-side, of course, is that you can't use a dumb client anymore; your client now needs to know things like how to get to an account given just an account number.

And just generally, why put more work on the client, when you can do it on the server? Well, server performance is something we're always trying to optimize - why NOT foist the work back to the client?

But let's also keep in mind that the Web 2.0 apps we know and love today aren't dumb clients. There's user-land code running there in your browser. Typically provided by the same server that's providing your resources in the first place. ie, JavaScript.

I realize that's a bad example for me to use; me being the guy who thinks browsers are a relatively terrible application client, but what the heck; that's the way things are today.

For folks who just want the data, and not the client code, because they have their own client code, well, they'll need some kind of description of how everything's laid out; the data within a resource representation, and the locations of the resources themselves. But the server already knows all that information, and could easily provide it in one of several formats (human- and machine-readable).

As an proof point of all of this, consider Google Maps. Think about how the map tiles are being downloaded, and how they might be being referenced as "links". Do you think that when Google Maps first displays a page, all the map tiles for that first map view are sent down as URLs? Think about what happens when you scroll the map area, and new tiles need to be downloaded. Does the client send a request to the server asking for the URLs for the new tiles? Or maybe those URLs were even sent down as part of the original request.

All rhetorical questions, for me anyway. I took a quick look at the JavaScript for Google Maps in FireBug, and realized I've already debugged enough obfuscated code for a few lifetimes. Probably a TOS violation to do that anyway. Sigh. I'll leave that exercise to younger pups. But ... what would you do?

For Google maps, it's easy to imagine programmatically generating the list of tiles based on location, map zoom level, and map window size. Assuming the tiles are all accessible via URLs that include location and zoom level somewhere in the URL. In that case, the client code for calculating the URLS of the tiles needed is just a math problem. Why make it more complex than that?

I think there are problem domains where dealing with 'links' as just data, instead of explicit URLs make sense, as outlined with Google Maps. Remember what Roy wrote in his presentation: "machines can follow links when they understand the data format and relationship types". Of course, there's plenty of good reason to use continue to use URLs for links as well, especially with dumb-ish clients.

the envelope, please

Yaron Goland posted a funny Star Wars -flavored story on jamming square pegs in round holes with Atom. There's a great conversation in the comments in Sam Ruby's reply.

I think it's fair to say there's a tension here; on the one hand, there's no need to wrap your data in Atom or protocols in APP if you don't need to - that's just more stuff you have to deal with. On the other hand, if Atom and APP support becomes ubiquitous, why not take advantage of some of that infrastructure; otherwise, you may find yourself reinventing wheels.

I can certainly feel Yaron's point of "I'm running into numerous people who think that if you just sprinkle some magic ATOM pixie dust on something then it suddenly becomes a standard and is interoperable." I'm seeing this a lot now too. Worrisome. Even more so now that more and more people are familiar with the concept of feeds, but don't understand the actual technology. I've seen people describe things as if feeds were being pushed to the client, for instance, instead of being pulled from the server.

One thing that bugs me is the potentially extraneous bits in feeds / entries that are actually required: atom:name, atom:title, and atom:summary. James Snell is right; it's simple enough to boilerplate these when needed. But to me, there's enough beauty in Atom and APP, to really make it reusable across a wide variety of problem domains, that these seem like warts.

Another thorn in my side is the notion of two ways of providing the content of an item. atom:content with embedded content, or linked to with the src attribute. The linked-to style is needed, clearly, for binary resources, like image files. But it's a complication; it would clearly be easier to have just one way to do it, and that would of course have to be the linked-to style.

The picture in my mind is that Atom becomes something like ls; just get me the list of the 'things' in the collection, and some of their metadata. I'll get the 'things' separately, if I even ever need them. Works for operating systems.

Of course, the tradeoff there is that there are big performance gains to be had by including your content IN the feed itself, with the embedded style; primarily in reducing the number of round-trips between client and server from 1 + # of resources, to 1. It doesn't help that our current web client of choice, the web browser, doesn't provide programmatic control over it's own cache of HTTP requests. Maybe if it did, the linked-to style would be less of a performance issue.

I suspect I'm splitting hairs to some extent; one of my many vices. I'm keeping an open mind; I'm glad people are actually playing with using Atom / APP as an application-level envelope / protocol. It's certainly worth the effort to try. There's plenty to learn here, and we're starting to have some nice infrastructure here to help us along, and the more people play, the more infrastructure we'll get, and ...

Tuesday, October 09, 2007

web and rest punditry

  • In "Google Map 'Documents'", Stefan Tilkov notes:

    In a comment, Patrick Mueller asks whether I'd consider Google Maps "document-oriented".

    Given that I can say that this is where I live and this is where I work I'd claim it's document-oriented enough for me :-)

    Stefan is referring to a comment discussion in his blog post on Lively. How we got from Lively to whether Google Maps is "document-oriented" ... well, read the original post and the comments.

    w/r/t "document oriented", the term is used on the Lively page, but I think Stefan is misinterpreting it. I can only infer from Stefan's post that he believes a web application is "document oriented" if you can generate a URL to somewhere in the middle of it. Like he did with two map links he included.

    I'd refer to this capability as having addressable pages. Addressable in terms of URLs that I can paste into a browser window, email to my mom, or post in a blog entry. Important quality. It's especially nice if you can just scrape the URL from the address bar of your browser, but not critical, as Google Maps proves; a button to generate a link is an acceptable, but less friendly substitute. Being addressable is especially hard for "web 2.0" apps, which is why I mention it at all.

    My read of the Lively page's usage of "document-oriented" is more a thought on application flow. In particular, pseudo-conversational processing. Which is both the way CICS "green screen" applications are written, as well as the way Web 1.0 applications are written. Turns out to be an incredible performant and scalable style of building applications that can run on fairly dumb clients. The reason I infer this as "document-oriented" is that, in the web case, you literally are going from document (one URL) to another document (another URL) as you progress through the application towards your final goal. Compared to the style of imperative and event driven programming you apparently do with Lively.

    So, with that thought in mind, Google Maps is clearly not "document oriented". The means by which you navigate through the map is 'live'; you aren't traversing pages like you did in old school MapQuest (which, btw, is also now "live").

    But even still, given my interpretation of Stefan's definition, I'd say there's no reason why a Lively app can be just as "document-oriented" as Google Maps given his definition; that is, exposing links to application state inside itself as URLs. You may need to work at it, like you would with any Web 2.0 app, but I don't see technical reason why it can't be done. Hint: client code running in your browser can inspect your URLs.

    Back to Stefan's original note about Lively: "it might be a good idea to work with the Web as opposed to fight against it". I think I missed Stefan's complaints about Google Maps fighting against the web. Because if Lively is fighting against the web, then so is Google Maps.

    Lastly, a note that Lively is built out of two technologies. JavaScript and SVG, and it runs in a web browser. I'm finding it really difficult to figure out how Lively is fighting the web.

  • In "What is SOA?", Pete Lacey notes:

    Another problem with the SOA name is the "service" bit. At least for me, the term "service" connotes a collection of non-uniform operations. I don't even like the phrase "REST Web services." Certainly, SOAP/WS-*, CORBA, DCOM, etc. fit this definition. But REST? Not so much. In REST the key abstraction is the resource, not the service interface. Therefore SOA (and I know this is not anyone's strict definition) encompasses the above mind set, but includes SOAP and similar technologies and excludes REST.

    If you change Pete's definition of "service" to be "a collection of operations", independent of whether they are uniform or not, then REST fits the definition of service. Next, you can simply say the resource (URL) is the service interface, for REST. Just a bunch of constraints / simplifications / specializations of the more generic 'service' story.

    Sure there are plenty of other details that separate REST from those other ... things. But you can say that about all of them; they're ALL different from each other, in the details. And at 10K feet, they're all doing generally the same thing.

    As a colleague mentioned to me the other day, REST is just another form of RPC.

    I feel like we might be throwing out the baby with the bath water here. It's true that I never want to go back to the CORBA or SOAP/WS-* worlds (I have the scars to prove I was there), but that doesn't mean there's nothing to learn from them. For instance, the client-facing story for REST seems a bit ugly to me. I know this isn't real REST, but if this is the kind of 'programming interface' that we have to look forward to, in terms of documentation and artifacts to help me as a programmer ... we got some problems.

    I look forward to seeing what Steve Vinoski brings to the table, as a fellow scar-bearer.