Links

pmuellr is Patrick Mueller

other pmuellr thangs: home page, twitter, flickr, github

Saturday, August 18, 2007

response to jsr 311

Marc Hadley responded quite quickly to my blog entry on JSR 311. Excellent. I'm really digging the transparency. I'm thinking Twitter may have played a part in the quick response.

w/r/t the GPL vs CDDL licensing issue; sorry, I'm sure I saw CDDL on some of the pages, I should have included it in the blog post. I frankly am not all that familiar with the CDDL, but Dave Johnson got me curious about something, and sure enough, it looks like Apache allows CDDL licensed binary software to be shipped with Apache projects. Note that the link above is a proposal, but it sounds like Apache is already doing this in some cases with CDDL. Very cool.

As for the other issues Marc raised; it's silly to debate via blog posts; looks like the dev@jsr311.dev.java.net would be the most appropriate place to have the discussion.

But I can't resist making two notes.

From Marc: "On the Java ME side, the fact that 311 is primarily a server side API ... means it is less suitable for the ME environment anyway."

To quote from the Java ME Sun Products overview, "As CDC is not limited to just the wireless market, and expands into other emerging markets like digital television, telematics, multi-function printers, ...". Those sound like platforms that could make use of a server supporting RESTful web services.

From Marc: "Jersey will automatically produce a WADL description"

Wonderful. The point isn't even necessarily that you should make this part of the spec, but to ensure that this is do-able within the current spec's design. Because you don't want to find out later, after the spec has been finalized, that the design prevents something like generating a description. Sounds like you're doing that with the WADL generation. I'd like to see the same done with the client.

Friday, August 17, 2007

on JSR 311

Some thoughts on JSR 311 - JAX-RS: The Java™ API for RESTful Web Services. The description of the JSR, from the web page is: "This JSR will develop an API for providing support for RESTful (Representational State Transfer) Web Services in the Java Platform."

Here are a few related links:

Some of the positive aspects of the ongoing effort:

  • It's good to be thinking about some facility to help people with RESTful services, in Java.

  • This JSR seems to be much, much more transparent than most JSRs; published meeting minutes (though a bit skimpy), mailing lists, reference implementation work being done in the open, etc. Wonderful. Wonderful!

  • Even the current drafts of the spec are available, on-line, without a soul-sucking click-through license agreement! The most recent one is the "Editors draft - July 3rd, 2007 (PDF)".

  • The general approach as currently implemented / described by Jersey feels more or less right. Of course, I say that because I've been recently looking at doing something similar, without having seen JSR 311, and there's a lot of commonality between the road I was heading down, and the one JSR 311 is going down. Use of annotations to augment methods and classes which describe the services, specifically. Some of the annotations I had come up with were in fact the exact same names that Jersey uses!

OK, now for the bad news.

  • The licensing for the open source side only accommodates the GPL half of the universe. Leaving folks wanting something more like a BSD style license, to fend for themselves. Of course, that's not necessarily bad, it's always useful to have some competition. But it's also nice to have a single common implementation that everyone can use. There are many licenses that would be compatible with the GPL and acceptable to a wider audience.

  • Specific issues from the 2007/07/03 spec draft: Section 1.2 Non-Goals:

    • Support for Java versions prior to J2SE 5.0: The API will make extensive use of annotations and will require J2SE 5.0 or later.

      Read: Sorry, J2ME. Sorry, folks stuck on Java 1.4.

    • Description, registration and discovery: The specification will neither define nor require any service description, registration or discovery capability.

      Read: We'll figure this out later; hopefully we won't have to change anything in this spec once we start thinking about this aspect of the problem.

    • Client APIs: The specification will not define client-side APIs. Other specifications are expected to provide such functionality.

      Read: We'll figure this out later; hopefully we won't have to change anything in this spec once we start thinking about this aspect of the problem.

The licensing issue is bad enough that it's really going to force an alternate implementation to be developed. This might be something that Apache would typically do, but given the Apache / Sun disagreement on JCP issues, it's not really clear to me that Apache will ever be interested in working on JSR implementations again.

Another interesting wrench to throw into the gears is Eclipse. As of Eclipse 3.3, the Equinox project has been shipping Jetty, along with some goop to allow you to use OSGi within servlets, or start an HTTP server as an OSGi service. Adding RESTful service support to this story seems like a logical next step to me. Note that the existing JSR 311 non-goal of support for <= Java 5 support violates an OSGi constraint of running in their smaller (1.4.x-ish) environments.

Seems to me, if we're going to have to work on an alternate implementation anyway (to solve the licensing issues), we might as well solve some of the technical problems as well (J2ME / Java 1.4 support, service descriptions, client apis, etc).

And yes, I do know of RESTlet™ but have not looked at it extensively; the licensing and Joint Copyright Agreement and trademark issues are non-starters for me. It is also on a pathway towards becoming a JSR.

Anyone up for a "Battle of the RESTful frameworks"?

Sunday, August 12, 2007

Ruby Hoedown, day 2

Saturday was day 2 of the Ruby Hoedown 2007. I had to leave early and so missed "Using C to Tune Your Ruby (or Rails) Application" by Jared Richardson, and the final keynote by Marcel Molina, Jr.

Nathaniel Talbott, one of the conference organizers, mentioned before one of the sessions that the sessions are being recorded, audio and video, along with the slides presented, and will be available on the web shortly. With some kind of a Creative Commons license. Wow! I wish more conferences did that. I won't dive into details on the sessions below, since the info will be available soon, verbatim.

Sessions I did attend:

  • Building Games with Ruby - Andrea O.K. Wright

    Andrea discussed a handful of gaming toolkits for Ruby. Supporting everything from MUDs to 3D. A few of the toolkits were wrappers over SDL.

  • Lightning talks

    • test/spec - Clinton Nixon - layers an RSpec-inspired interface on top of Test::Unit
    • a strategy game written in JRuby, using Swing - didn't get the presenter's name
    • methodphitamine - Jay Phillips - a more powerful alternative to Symbol#to_proc
    • Ruleby - Joe Kutner - a pure Ruby rule engine
    • tyrant - Patrick Reagan - a way to run Rails applications without each user needing a development environment
    • generating Ruby classes - Luke Kanies - dynamic code gen
    • Iron Ruby - Brian Hitney - Ruby implemented on the CLR
    • static sites with Rails - Brian Adkins - how to enable mostly static pages with Rails
    • Thoughtworks test strategies - Dan Manges - how TW designs tests for teams
    • myDecisionHelper - Tyler Start (sp?) - self-help web app

  • BOFs over lunch

    Went to Rick Denatale's "Smalltalk for Rubyists; Ruby for Smalltalkers". More the former than the latter, unfortunately for me. But still interesting and informative.

  • Does Ruby Have a Chasm to Cross? - Ken Auer

    Great talk; like Bruce Tate's keynote on Friday, referenced the book "Crossing the Chasm" (just added it to my 'wanna read' list). Ken's talk came at it from a different angle: being there, when Smalltalk was just about to 'cross the chasm', but didn't make it. Why didn't it? What can we learn from Smalltalk's tragic history?

All in all, great conference. I'll be there next year, assuming they have it again. Maybe I'll have some more Ruby under my belt by then.

Forgot to mention in the previous post that I used my Nintendo DS-Lite with the Opera browser, again, to do some lite surfing, email checking, twittering, during the conference. Very convenient.

Saturday, August 11, 2007

Ruby Hoedown, day 1

theRab tweets: "i'm afraid to google 'ruby hoedown' so i'll just patiently wait for your blog post". Here you go. BTW, no need to be afraid of Google (at the moment).

The Ruby Hoedown 2007 is a small conference covering Ruby topics, held in Raleigh, NC. At Red Hat HQ, the same place where barcampRDU was held last week.

Friday was just a half day afternoon session, though there was a testing workshop in the morning that I didn't attend. A bit over a hundred attendees, I'd guess, many from out of town. Usual list of sponsors, with one surprise: Microsoft; though obviously that's not a complete surprise.

I recognized a lot of folks there from last week's barcamp, and some from the erlounge meet-up. Some "Old Dudes Who Know Smalltalk" were present, including my evil twin, Rick Denatale, and someone I hadn't talked to in ages, Ken Auer.

Sessions this afternoon:

  • Exploring Merb - Ezra Zygmuntowicz

    Merb is Ezra's pocket-framework for web serving. Lighter than Rails. Ezra stressed twice that Merb "doesn't use cgi.rb" - I'm showing my n00by-ness by assuming this is a good thing.

    Between this session, and the camping session at barcamp, there appears to be active and interesting development still in the web server framework arena, at the lowest levels. Like the days in Java, before servlet. Sigh.

    One cool feature of Merb is supporting direct streaming of content. For example, if you need to return the contents of a file, you just return the file handle of the file where you would normally return the content, and the file gets streamed on your behalf. Handles uploads too, and it sounded like a lot of people use it just for the upload capability. Ezra also mentioned someone using Amazon S3 as their store, and making use of this feature, where Merb was really just the gateway between the streams (to/from S3, to/from client).

  • Next-Gen VoIP Development with Ruby and Adhearsion - Jay Phillips

    General notion here is that setting up Asterisk is hard, because the configuration files are large, monolithic, and complicated. Adhearsion makes that simpler, as your configuration becomes small Ruby programs. Which maybe you could imagine sharing with other people ("Want my call-center scripts?"), compared to the situation today where granular sharing isn't really possible.

    I don't have much need to set up a VoIP box, otherwise I'm sure this would be fun to play with. Jay did mention that Asterisk has VMWare images available, which would make it even easier to play with, although Adhearsion is not current available on those images. If I only had more time.

  • Keynote - Bruce Tate

    Really good talk discussing how Ruby became as popular as it is, how Java became as popular as it was (is?), what has dragged Java down in recent years, and some future thinking and advice.

    Best piece of advice: "Java didn't start as something that was complicated, it evolved into something that was complicated. Ruby is not immune to this."

    SmallTalk was mentioned; yes, sadly, with the capital T; so Bruce wasn't one of us I guess, he at least knew it was good stuff. OTI was even mentioned.

    Bruce mentioned the ultimate plan for Ruby: TGD - Total Global Domination. Hmmm, that reminds me of something.

I asked whether the presentations would be available on the web, and the answer was yes. In addition, the sessions are being recorded (audio and video) and will also be available on the web at some point.

Looking forward to Saturday's sessions.

Sunday, August 05, 2007

barcampRDU 2007

On Saturday I attended BarCampRDU 2007, and here's a quick round-up.

First, thanks to the organizers and sponsors; it was fun geeky way to spend a Saturday, and the amenities were fantastic, especially considering the price of admission. Especially enjoyed the catered coffee bar and lunch from Neomonde.

Here's a list of some of the sessions that were run:

  • Juggling
  • Distributed SCM - Bazaar, GIT, etc
  • Camping in 10 (Ruby)
  • Linux virtualization
  • Bug House, a chess variant
  • Podcasting
  • Ruby and X-10
  • TDD
  • Extending SketchUp with Ruby
  • GTD
  • OpenMoko
  • Grails
  • Geeks for Good
  • Startup Weekend
  • What's stopping you from doing something?
  • Open Services
  • OLPC
  • Digital Identity Management
  • FaceBook application development

Lots o' pictures here, and lots o' blog entries here.

Some of the sessions I went to:

Camping at 10
Very cool; I really need to dive deeper into Ruby. Completely random factoid: of the 15 laptops in the room for this session, 12 were Macs.

FaceBook application development
Quite illuminating. I had thought all those 'user generated' FaceBook apps were actually hosted by/on FaceBook, but they aren't. Which makes me even more leery of what those applications are doing with the data we're giving them. hmmm

Open Services
Whereas Open Source / Free Software has defined freedoms surrounding source code, the next battleground on the freedom front are the services we use on the web. What does it mean for something to be an "Open Service"? Not sure why Luis thought he "trivialized the discussion"; absolutely not. Lots of really interesting things to think about here. Especially as we find our data getting mashed up and otherwise reused in ways we never expected.

Other notes ...

Rather than take my laptop into the unconference, I decided to try living with just my Nintendo DS-Lite with the Opera browser. I've surfed a bit here and there with it, but not for an entire day. While it's quite capable at handling degraded 'mobile' sites, available for Google Reader, GMail, and Twitter, for instance, it's rather painful for websites designed for desktops. Most folks probably won't tolerate the slow, small device, especially compared to the iPhone. But having worked on embedded devices for years, I can easily overlook the limitations. It's a lot cheaper than the iPhone also.

All in all, worked out pretty good; checked my mail, checked twitter, checked blogs; posted a few twitters. A bit painful, pecking out messages on a tiny touch screen with a stylus, but worth it to not have to lug my MacBook around in a backpack. Just had to lug around my man purse.

Lastly, one thing I already knew, but had reinforced, was that all the kewl kids wear bright yellow / orange shirts from Threadless.

Friday, August 03, 2007

how RESTy need we be

A few blog posts reverberating in my mind:

I just ran across a post by Joe Gregorio today, where he's comparing WS-* RPC vs. REST, specifically talking about the fact that WS-* RPC only uses the POST request method of HTTP:

"That POST of a generic media type gives no indication if the request is safe, or idempotent, nor is there any indication of the cachability of the response."

I also just read this morning a post by Leonard Richardson on Amazon's new FPS service, commenting on the 'REST'y ness of the service:

"its 'REST' interface is about as RESTful as Flickr's and del.icio.us's 'REST' interfaces"

Note, the Flickr and del.icio.us aren't considered terribly RESTy. :-)

Joe notes a handful of things that "just using POST in HTTP" breaks. Leonard notes that FPS isn't truly RESTy, but is in fact just as non-RESTy as two other fairly popular services.

And I'm left wondering: do we really need to do everything in pure REST?

Perhaps we can identify a small set of characteristics that get us most of the benefits of REST, that would be easier to implement than going 'full' REST. And just focus on those. Because apparently, it's hard to do full REST. Or to be even more pessimistic, perhaps there are disadvantages to using REST. Why would Amazon and Yahoo not use REST?

Most of the obvious advantages with REST revolve around the GET verb. Make sure you only put safe, idempotent actions behind the logic of your GET processing. Use ETags, or even better, cache directives, in your HTTP responses on the GETs to allow clients to intelligently cache things.

What else? Are there really other benefits? If there are, why are people still not jumping on the RESTy bandwagon?

Here's my thought on the lack of adoption of REST: it's complicated for anything but the GETs. Namely, the mutating verbs POST, PUT, and DELETE. Not technically, but semantically. Check the rest-discuss mailing list sometime. I've been having conversations with folks here recently regarding some of the semantics of 'updates' as well; ask 5 people their thoughts, you'll get 10 different answers.

Martin Nally had told me a few months ago that he thought people should be able to do 90-95% of their services in a RESTy style, and for those interfaces that didn't fit the REST mold, you could do 'em in non-RESTy style (ie, some kind of RPC-ish non-GET mutation operation), but it should cost you a $1000. I think I'm ready to lower the price to $100.


Updated on 2007/08/03 to fix a syntax error in the image element.

Thursday, August 02, 2007

client generation

Dan Jemiolo: What kind of client generation are you looking for?

~shiver~

I suppose I must not have gotten around to telling Dan my horror stories of using WSDL in the early, early days of Jazz. The low point was when it once took me two working days to get the code working again, after we made some slight changes to the WSDL. Of course, we were doing some evil, evil things, like parsing Java code with a JDT jar in an ant task, and replacing generated code from the WSDL generation process. But still, code generation of this ilk leaves a bad taste in my mouth.

Also see Dare's issues with bananas.

The best code generation is no code generation.

And that's what we changed in Jazz. Because we already had complete control over the data typing story (EMF), we had no problem generating the XML and JSON we wanted, completely dynamically, by reflecting over the data used in our services. But we had to do something about the service methods themselves.

So we rolled our own client and server side stack for this.

We kept the notion of defining the operations in a web service in a Java interface, because this makes a lot of sense to do in Java. We can reflect over it to look at the methods and signatures. On the server, you can write a class to implement the interface, and that's your service implementation. The low-level server interface (ie, Servlet for Java) can figure out what service to invoke, and then call the service implementation reflectively. And on the client, you can use Proxy and friends to build an object which implements the interface by making the HTTP request on your behalf.

(Quick aside to note that Jazz services are largely RPC styled, though there are some that are more RESTy flavored - stay tuned; they've caught the REST bug. I think the 'client library' or invocation style is largely independent of the architectural style, so I think everything I'm saying here completely holds for REST as well as RPC, and everything in between.)

By having the client and server work completely reflectively, all we had to do was make sure the data classes and service interfaces were the same (or at least compatible) between the client and server. Understand, "all we had to do" can be a problem; but at least we didn't have the generated code to deal with as well, nor did we have a separate build step for it that can be an agility killer.

It goes without saying that you can get by with a lot less configuration in such a situation. Reflection over Configuration. Don't Repeat Yourself.

Looking back on this, I think this was a great trade-off in the time expended to build the stacks. For instance, we were able to tweak the system in various ways that would have been impossible to do with a code-gen built system. I suspect this is going to be the case for any medium- to large-scaled system built using a number of services. You can either lock yourself into a system under which you have very limited control, and spend your time working around it and fighting it, or you can write code customized to your needs and spend time tweaking as you need.

Let's get back to your original question, but tweak it a bit: What should we do to make it easier for people to build clients?

  • Provide machine-readable descriptions of the service interfaces. These machine-readable descriptions can be used to generate human-readable descriptions. For instance, instead of generating an HTML table for your Gregorio tables, how about generating JSON? Then you can generate an HTML file that uses some JavaScript to format the result in an HTML table. Perhaps you can use it, once you've built a little client invocation stack, to build some dynamic callable objects which serve as proxies to the server (if you're lucky enough to be working in a dynamic language).
  • Provide machine-readable descriptions of the data flowing over the wire. Not just the HTTP content, but query string parameters where appropriate.

If you can reflect on your service interfaces and data dynamically in the server, then you can generate all this meta-data reflectively as well.

Or if you really do want to do some code gen, and your client is able to load directly runnable code from your server *cough* JavaScript *cough* then it's easy to imagine that you could code-gen a client library dynamically on the server as well. I suspect that's actually overkill though.

Tuesday, July 31, 2007

REST helpers

Dan Jemiolo posted today about his restdoc tool, which produces Gregorio Tables from comments in REST service implementations designed to be run in Project Zero. He also posted to the Project Zero forum, and included some screen shots of the output of his tool, here.

Here is what's cool about this:

  • I love it when I can keep artifact information like this with my source code; easy to keep in sync.

  • It's genuinely useful information.

  • I'm tired of the "we don't need no stinkin' tools" attitude of some of the RESTafarians. Baloney. I think we can certainly live without overengineered tools like WSDL, but having small tools can certainly help.

Now, for some rocks:

  • As I mentioned in a comment on Dan's blog, it would probably be useful to have this information available at runtime on the server; the server could actually validate what it's doing. And I'm not talking about the server reading it's restdoc information back in; I'm talking about the server having that information available to it, at runtime, obtained reflectively. The advantage is that you know the information is never stale. No more, "When was the last time I ran restdoc again?". In Java, you would presumably use annotations to do this.

  • While Joe's tables included a non-precise description of the HTTP content for the method / uri rows (eg, "Employee Format"), restdoc only includes the 'format' (eg, JSON). I'd prefer to see more precise typing, but to start with, something as vague as Joe's would be good.

  • That HTML table looks like it may be too wide to be viewed on an iPhone. The good news is, I don't have to worry about that problem.

Finally, not to knock Dan at all, but I really have to wonder what's going on if the best we can do is describe our services in human readable tables of text. Really? I mean, can't these multi-core, multi-gigahertz computers of ours really help out anymore than by rendering some HTML to a display that we read while hand-coding our HTTP invocations?

That just isn't right.

Friday, July 13, 2007

erlang meet-up

I went to the first erlounge RDU meet-up tonight to find out more about Erlang. The meet-up was arranged by Kevin Smith, who provided a short presentation.

I had done a little boning up on Erlang last night, so the presentation wasn't completely foreign to me, and I was able to squeeze a few questions in while the S5 presentation took forever to switch slides. Some of those questions:

  • Q: Does Erlang compile to binary, or use bytecodes, or ???

    A: Compile to bytecodes.

  • Q: What's the deployment story for web server code?

    A: Run yaws, a web server written in Erlang. Presumably, you'd be able to proxy to this via Apache, like other stand-alone servers (on a shared host, for instance; TextDrive doesn't currently have it installed, asking about it now).

After doing a bit of reading last night, I came away a little less enthusiastic than when I started. Not sure why. The meet-up re-invigorated me a bit. It's probably time to buy the book. Even if I don't actually do anything with Erlang, it's always nice to see what's going on with other languages, in hopes of maybe transferring some of those ideas, as appropriate, into other work you're doing.

On the negative side, it sounds like the error reporting (compilation and runtime) is fairly nasty. And the doc is largely man pages. And there's no real unicode support. And it's compiled.

It was a good omen to see my old young buddy, and fellow Building-50x-ite Chris Grindstaff at the meet-up. Chris has been yammering on about Ruby for years. Coincidently, the erlounge RDU meet-up group is an off-shoot of the Raleigh-area Ruby Brigade. Hmmm.

Monday, July 09, 2007

a history of transparency

Two projects in IBM have recently decloaked, both which I've had the pleasure of working on: Jazz and Project Zero. Both projects have come under some attack by folks as being "not open source". For instance, see Mark Pilgrim's typically humorous response to Project Zero.

I'm not interested in talking about that specific aspect of the projects. I'm an open source commie from way back, but the fact of the matter is that IBM contributes a lot of open source to various communities, and at the same time produces commercial, proprietary software.

What I'm quite happy about with both products is the transparency they provide to IBM's product development process. The term "transparent" I've borrowed from Stephen O'Grady, which he used to describe the Jazz development process. I thought I'd give a quick rundown on the transparency I've experienced in my 20+ years of software development at IBM.

From 1985 till about 1995 I worked on projects internal to IBM, and didn't really have a need to deal with customer feedback on the products I was working on, because they were internal to IBM. We did have a 'newsgroup'-like system called "IBMPC Conferencing" that was a fantastic resource for IBMers. Not a whole lot of people used it (relatively), which was unfortunate, but also kept the wheat/chaff percentage quite high. It did serve as a way to communicate with our internal customers though.

The IBMPC conferencing system expanded sometime in the early 1990's to allow customers access to a restricted set of 'newsgroups' called CFORUMs. This was quite convenient for IBM developers, since it used the same news system we already knew how to use, and customers also had access to it somehow. Sometime later, IBM created a real NNTP server named news.software.ibm.com, which served the same purpose as CFORUMs, but used the more 'standard' NNTP protocol.

By this time, I was working on VisualAge Smalltalk, and we created at least one newsgroup for it on the NNTP server. For other projects I was involved with later, we also created newsgroups on the server.

So, we've provided some kind of newsgroup-y access to product groups for a while. However, we were limited in what we could actually discuss on the newsgroups. While we would frequently accept bug reports from users posted to the newsgroups, we couldn't really provide bug tracking numbers, since there was no way for users to access our bug tracking systems. We did it anyway, to at least give our users a shorter handle by which to reference the bugs. For new feature requests, or requests for when the next release would be made available, you'd typically see a response of "We cannot provide information regarding future versions of the product". Which was insanely frustrating for us developers. But we bit our tongues and pasted that response into posts, a lot.

So now, roll forward to the mid-late 2000's, with Jazz and Project Zero. We have 'newsgroup'-y access like we have had for a while. But we also have access to the bug tracking systems. And source code management systems. And a general notion of talking more openly about future release content and dates (I hope!)

It certainly seems to me, that over time, we've become more transparent with our development processes for our commercial products. I think this is great for customers, who will have more direct access to the product development teams and the product development process itself. It's also great for the development groups within IBM, as these open processes are a great equalizer: both new hires and CTOs have the opportunity to converse equally with customers.

I'm a firm believer in this sort of transparency being a benefit to everyone, and I'm looking forward to this becoming the rule rather than the exception for more IBM products.