Links

pmuellr is Patrick Mueller

other pmuellr thangs: home page, twitter, flickr, github

Tuesday, August 29, 2006

Unix Power Tools circa 1993

So as I was reading some of the textmate manual last night, I happened to think ... 'I really don't know the bash shell scripting language at all!'. And that I really should make an effort. Since I use it everywhere (except Windows, but I try not to use Windows). So I went looking through my bookshelf ... have I ever bought a book on shell programming? Like Bill, I suffer from a problem of buying too many books. * I figured I probably had bought one at one point. And sure enough, I found a book on the Korn shell book from O'Reilly. But I didn't want to try to figure out what the difference between the Korn shell and Bash were. So I kept looking.

And then I found some buried treasure. "Unix Power Tools", First Edition published by O'Reilly in 1993. Huge book. 1100 pages. No longer in print, but here's the modern version. What's so cool about this book is the peek it gives us on what the 'unix' world was like back then. Here are some reminders:

  • There are no references to Linux, anywhere. Nor BSD (by itself). A CD of utilities is provided with the book, with precompiled binaries for "seven of the most common Unix platforms":
    • Sun4 SunOS 4.1.1
    • Sun3 SunOS 4.1.1 (is Sun3 vs Sun4 a hardware statement?)
    • DEC Ultrix 4.1
    • IBM AIX 3.2
    • HP-UX 8.07 (what a big version number!)
    • SCO XENIX 2.3.2 (shudder)
    • SCO UNIX 3.2.x
  • For word processing, use nroff / groff
  • For version control, use sccs or rcs (aside: I have a funny story about Walter Tichy, a prof of mine at Purdue, who wrote RCS; it will cost you one measily beer to hear it!)
  • "A summary of common UNIX networking and communications utilities:
    • The Internet (I will spare you the description)
    • mail
    • ftp
    • uucp
    • telnet
    • other usual stuff rlogin, rcp, rsh, NFS
  • Number of pages describing these common UNIX networking and communications utilities: 1
  • Number of pages spent on vi: 52
  • Number of pages spent on emacs: 14
  • Number of pages spent on awk: 36
  • Number of pages spent on perl: 10

No Linux, no BSD, no python or php. No web, no http. Not even gopher. No ssh.

My god ... the dark ages.

It's easy to poke fun at some of this, but honestly, the book still has some value. Chapters on find, sed, etc, that I'm sure I'll read again some day, and understand the tool for a short while before forgetting again. How often has find really changed in 13 years, anyway?

It also has the traditional huge O'Reilly index, which is nice. And the chapters are split into sections, and both the page number, chapter number, section number are on ALL the pages. If there's one thing I hate, it's books that use numbered sections, they do intra-book linkages using the section numbers, but the pages don't have the section numbers.

So I think I'll keep it rather than sell it at our next yard sale, or otherwise dispose of it. And leave it sitting around for a while to read up on find again.

* psss ... if you suffer this debilitating book buying disease, here's at least one step you can take; stop buying non-technical books and use your local library for those, assuming you have one. And look into BookMooch (on my list of things to do).

Monday, August 28, 2006

Programmable Text Editors

I have to admit that I'm a sucker for programmable text editors.

My illness began back in the early 80's when I was a co-op at IBM, working on an IBM mainframe. We used XEDIT as our text editor. XEDIT had a 'macro' language, which happened to be the same 'macro' language used by the shell, REXX. Our application program was written in REXX, so once you knew how to program it, and figured out the trivial interface to writing macros for XEDIT, you could really make that sucker fly. In fact, most of the interesting GUI programs that ran on VM/CMS at the time were implemented with XEDIT as the GUI interface.

One of the first programs I bought for the first computer I bought, a PCjr, was Manfield's KEDIT, pretty much a complete clone of XEDIT, along with a pretty decent implementation of REXX. (Also had Turbo Pascal and Smalltalk/V running on that puny thing).

Fast forward a few years and I found myself in the IBM Cary Regency lab (aka "The Country Club") working on a text editor for IBM. That never saw the light of day. Except as the GUI engine behind my graphical rexx debugger RxD. My job on the editor team was to expose the functionality of the editor in REXX. You could write macros in C, but who wants to do that.

Fast forward a couple of years and while working on something where I was on AIX all the time, I spent the time to learn Emacs and even write some macros.

Fast forward a couple of years and while working on Smalltalk I found a few hook points in the ETWindow hierarchy so I could customize the basic text editor. Since the one provided was quite bare-bones. Customize it with Smalltalk, of course.

Things have never been quite the same since. I still use KEDIT to this day, for certain tasks, on Win32. But more often than that I'm in Crimson Editor. Pretty much stick to vi on Linux (my arm position when I'm couch coding doesn't lend itself to all the ctrl/alt/shift madness of Emacs). On my mac, where I spend most of my time, I use BBEDIT. And for anything half-way serious I'm usually in Eclipse.

One of the common denominators for these environments (save KEDIT) is that they aren't programmable. Or, at least, easily programmable. Nothing like writing macros in REXX, Lisp, or Smalltalk anyway.

So, I was quite excited when reading the manual for TextMate, that they have a macro programming capability built in. And the language is ... anything you want. Basically, shell scripts; so stick a #!/usr/bin/env ruby at the top of your macro and write it in ruby. They even support back-tick 'include the output of this command' in the code templating system.

I got a new toy.

Friday, April 28, 2006

A Reel Man making the rounds

My buddy Todd Lothery's movie, A Reel Man, is making the rounds. Here's a short blurb I pulled from the Indianapolis International Film Festival:

A Reel Man is a profile of Skip Elsheimer, a Raleigh, North Carolina resident who collects educational films. Skip started collecting these 16mm films many years ago, and he now has almost 17,000. His archive includes films from the 1940s, ‘50s, ‘60s, ‘70s and ‘80s on every conceivable instructional topic – sex education, personal hygiene, public safety, driver’s ed, occupational training, the dangers of drugs and alcohol, how to be popular, etc. The films in Skip’s archive are campy, amusing and not a little propagandistic, but they’re also valuable, fascinating time capsules that reflect America’s social attitudes and cultural values throughout the last half-century.

A Reel Man appears to also have been shown on a public tv network in South Carolina.

I saw A Reel Man as Todd's project at the Center for Documentary Studies at Duke, where he honed his craft. The movie had it's first public showing at the Asheville Film Festival.

Cool.

For Raleigh-ites, you may remember Todd as a movie reviewer for the News and Observer a few years back. He and his wife Kay Wiles headed back home to Indy to be closer to family. We miss 'em!

Also for Raleigh-ites, Skip's A/V Geeks web site has a list of events where he'll be showing his movies, including free showings fairly often at the NC Museum of Natural Sciences.

For everyone, A/V Geeks has lots of stuff available for purchase, and some downloadable movies.

I gotta get Todd to upload his movie to YouTube after it makes the rounds.

Monday, April 24, 2006

The problems with JSON

My love affair with JSON is over. I'm going to outline the problems I've got with it, and then also reiterate some of the things I do still like about it.

  • Licensing
    I will admit this point is rather weak. It's based on the license that Douglas Crockford ("father of JSON?") uses for the JSON code he makes available. It's an MIT-ish license, I suppose, but has an odd additional stipulation in it. The Software shall be used for Good, not Evil. There is some discussion of the license here (scroll about half way down the page). My experience dealing with IBM lawyers for the past 15 years, is that this is exactly the thing that drives them nuts. I'm with Mark on this (read the blog post), and I'll admit that when I see stuff like this, I just immediately assume a battle ahead with lawyers trying to get permission to reship it. I'm getting too old for that shit.

    So, like I said, this is a weak point, since as Crockford himself says, JSON parsers and generators are easy to write. It's just ... a shame.

  • Lack of complex data types
    While JSON has great support for basic data types, unlike XML, it has no built-in support for complex data types. In JSON, instead of objects, you really have hash tables, where the properties/attributes of the 'object' are string keys, and the values are any of the valid JSON objects, including an 'object' (hash table).

    Unfortunately, for my purposes, this isn't good enough. The objects I'm dealing with right now at work are actually classically designed object-oriented instances of classes. That can be subclassed. JSON has no built-in way to indicate the actual type of the object, just the properties on the object. Every object is 'anonymous'. Or again, just a hash table.

    You can kind of fix this by including a type name as a property of the object, but this is a bit of a hack. And it's non-standard. And if you actually looked at some of the stuff we were JSON-ing, you'd realize that you'd really like to have something like XML namespaces to help cut down on string duplication, if your type names are long (ie, they include an XML namespace, package, module name). That's hard to fix.

    XML on the other hand, does a pretty good job of this. Type names can end up becoming actual element names, using namespaces to end up with a short form of the longer, universally unique class name. With the long name declared up front, kinda like a Java import statement. Nice.

  • Executable
    The end of the day, this is really the biggest problem. This is executable code. How long have we known that you really shouldn't be sending executable code over the network; you gotta have a lot of trust in your partner if you're going to execute code they give you. With JSON in JavaScript, you really should forgoe using eval() or any other function using eval(), unless you are really sure you can trust your partner. And I would claim that's hard. Ask MySpace.

    Instead of using eval(), you should really use some code which actually parses up the JSON text. Like every other language does. But, you gotta wonder, what's faster, a JSON parser written in JavaScript, or the existing DOM engines to parse XML? My bet's on the DOM engines. Complete guess. But even if for some reason a JSON parser was faster than XML, it still means you have to carry around this wad of JS code to parse JSON for your pages. Baggage.

So, what do I still like about it?

  • Support for basic data types
    I still really love the self-descriptive typing for basic data types in JSON. Compared to XML, where just because you see a text node or attribute value of "true", you really have no idea if that this is a string, or a boolean, or maybe even some specialized marshalling of another type. No problem with JSON; "true", the string, is rendered in a completely different way than "true", the boolean. This is a fantastic property of JSON; hopefully future structured serialization languages can likewise incorporate this.

  • Easy for JavaScript users
    Really easy. My friends using dojo said that handling dojo from REST calls is quite simple. Anything to make life a little easier in that world is nice. If nothing else, using JSON for prototyping is probably a great idea.

Tuesday, April 18, 2006

Debugging 101

Great post over at Hacknot about debugging: "Debugging 101". I've been doing this crap for 20 years, so it's not like any of this is new, but it's certainly interesting to see it all written down in one place, and even though I've done it all, I don't neccessarily remember it every time :-)

A couple of highlights/issues:

Wooden Indian: I can't remember how many times over the last decade or so I've done this with Tim Wolf, and him with me, back when I was working at the OTI building in Raleigh. It just always seemed to be so funny to be describing a problem to him, when he really had no idea what I was working on in general, and just a few interesting questions from him, and the lightbulbs start going off in my head. And vice versa. And of course, asking lots of questions is an important way for junior folk to learn from their elders. Hint: I know I ramble; don't be afraid to call me on this.

Avoid Debuggers: I take issue with this. To me, that's like saying don't use them fancy compilers; they're just a crutch; you should write all your code in ASM! Perhaps learning to use a debugger properly is the key. Here are two of Mr. Ed's points:

  • How often does it happen that you spend an extended period of time carefully stepping through a piece of code, statement by statement, only to find at the critical moment that you accidentally "step over" rather than "step into" some method call, and miss the point where a significant change in program state occurs? In contrast, when you progressively add trace statements to the code, you are building up a picture of the code in execution that cannot be suddenly lost or corrupted. This repeatability is highly valuable - you're monotonically progressing towards your goal.

    It's not like adding code to the problem isn't going to cause it's own problems; not that it always does, but it sometimes does; and unless it's actually logging code you're adding, you're going to have to remove/comment out/neuter that trace code before you back into your team repository (or production). Which you will someday forget to do.

    In fact, I frequently debug exactly like the tracing description; add breakpoints to help narrow down the interesting points where I want to watch what's happening.

  • I've noticed that habitual use of symbolic debuggers also tends to discourage serious reflection on the problem. It becomes a knee-jerk response to fire up the debugger the instant a bug is encountered and start stepping through code, waiting for the debugger to reveal where the fault is.

    That's just silly. It's a tool. Not a magic eight ball. Folks who wait for the debugger to show them the problem quickly stop using debuggers, since debuggers don't do this.

It's not like a nice 'graphical' debugger will save your ass every time. There are plenty of other things that you will sometimes need to do (read the article!). But I'd be screwed if I couldn't use a nice debugger like Eclipse's Java debugger for debugging the incredibly complicated Java code that I live in. In fact, I typically do my 'launches' under debug mode; bonus with Eclipse (at least when using a J9 VM, not sure about the Sun VM) is you get tasty hot code replace when you run under debug.

I have a long history with 'graphical' debuggers. When I started my IBM career I was doing CICS application programming in PL/I. The debugger the system shipped with, CEDF, would only pause before and just after your system or database calls. You couldn't step through your code at all. Even see it. Shortly before I moved onto my next job, we got a trial usage of a product called Intertest (which appears to still be around). Oh. My. God. Stepping through code, a line at a time. Inspecting local variables. It was like freaking magic. It costs an arm and a leg, but I managed to convince the folks on site that we absolutely needed this.

Later, when I was one of those OS/2 hippies, I wrote RxD, which I think was using the coolio text editor control that was the day job. That was fun. Nothing like actually writing a debugger.

In Smalltalk, it was fairly easy to hook into the debug system, so I did some whacky stuff like audio break points; set a .wav file to play when a statement was hit. Nothing like listening to your code 'play'.

Man, am I old, and a geek.

RIP: Kim Clohessy

Got some sad news this weekend. Kim Clohessy passed away. Mike Milinkovich also covered this today. +1 on everything said there.

I first met Kim when I went on assignment from IBM to OTI in 1997. I had specifically started working with the embedded software group in Phoenix, and Kim was one of the technical/executive leads there. Kim was one of those great executive and technical guys; worked great in front of customers selling our stuff, but he also knew our stuff. Hell, he helped invent it! But you know, there just aren't enough folks like that.

Kim is actually the only person I personally know who wrote a practical app for Palm with the J9 VM, which was our baby back then; some kind of farm/ranch management tool for his horses, IIRC. He was obviously proud of it, I remember him mentioning it more than once. Of course, he deserved to brag, J9 on the Palm wasn't the easiest thing to work with in early days. You pretty much had to be a rocket scientist.

The picture on Mike's blog is a great picture to remember Kim by; smiling like he usually did.

Later, dude.

Wednesday, April 12, 2006

Content-Type is dead?

This meme has been floating around today. Here are some links, on both sides of the issue:

Dang! And we just added some code this week to optionally returns JSON or XML, depending on the HTTP Accept: header, from a REST service. It just seemed so ... natural. Yahoo! is doing it via a common query string parameter, which I guess you could say is the Lo REST way, whereas the Accept: header is the Hi REST way.

So I can see that maybe Lo REST is a more portable way to do this than the Hi REST way. Can I really set headers in my http client? Am I sure it's not going to futz with them? Maybe some proxy will lose them, or someone will reuse the URL without remembering to set the Accept: header. Query string parameters ALWAYS work.

That, I can buy, as an argument for not using Accept:. Or maybe, as an argument to allow both Accept: and the query string parameter (who wins?).

Anyway, I decided to check the trail on this a little more, by reading the notes that led up to Larry Masinter's response. OK, the cellmt guys, with their Accept: header of

Accept: application/cellml-1.0+xml; q=0.5, application/cellml-1.1+xml; q=1

is ... yeah ... a little over the top. Does anyone really do anything on the server with the q parameters? And those mime-types, with the version built into them ... whoa. *and* the "+xml" !

Then I read up on HTTP 1.1 Content Negotiation. Ok, sure. No one does any of this stuff. If I had waited just a few hours more, I could have used this to put me to sleep.

So Larry is right, "HTTP content negotiation was one of those "nice in theory" protocol additions that, in practice, didn't work out."

Ian is wrong, Content-Type still has value, although maybe it has none in a web browser rendering web pages. I don't really care.

Dare makes a little sense: I do think some Web/REST advocates need to look around and realize what's happening on the Web instead of arguing from an "ideal" or "theoretical" perspective. My take-away: use Lo REST vs. Hi REST. Dare also makes no sense, seemingly comparing the state of HTML and multi-media rendering (sucky support for content type) to web services via REST. They both use HTTP. And ... maybe that's about it! So the web browsers screwed up content type, that's their fault; why can't the RESTafarians try to do it right, if they want?

Wednesday, April 05, 2006

Triple XXX

BoingBoing ran a story today on Triple XXX, a restaurant in West Lafayette, IN, which is where Purdue is located. Which is where I went to college.

Very popular place to go late at night, after drinking way too much beer, to get a burger and onion rings. Lots of folks liked the Duane Purvis, including Sandy, but the peanut butter was a little over the top for me. I usually got the Bert Burger, or maybe eggs, or sausage gravy, depending on how close to dawn it was (or past).

My senior year, I rented a house with 6 friends, on Littleton Street, across from the hugemongous frats, about 5 houses down from Triple XXX. It was very convenient. I remember at the end of fall semester, no one had enough money to pay for a big shopping trip, so we just stopped shopping. And ate at Triple XXX a lot; except on Tuesdays; 25 cent tacos at Taco Bell on Tuesdays.

In the picture above (click the picture for a slightly larger version), circa 1960, you can just barely see the sign for the Clark gas station under the car cover. That's where we'd buy our cigs. I was there in the 80's; besides the cars, it looks pretty much the same.

I think we used to call the restaurant Tri Chi, but looking at the name now, it's funny that it's called Triple XXX, when we pronounced the name "Triple X". Not "Triple X X X". Which would have been way too long. Wouldn't Triple XXX be XXXXXXXXX?

I love in the picture above they have to make a point to show the restaurant is not an 'adult' restaurant (whatever that might mean).

Gonna have to get me a shirt!

Monday, April 03, 2006

Comments enabled

I decided to enable comments; previously, I had specifically disabled them. I'm not a big fan of commenting in blogs, because without a nice interface to let me track who responds to comments I make, it's not really worth it. I really don't want to have to go back to someone's blog web site to check. I'm sure that will change some day.

In the meantime, I got a couple emails this weekend from some folks at IBM concerning my posts on VMware. I figure I might as well be a little nice about it, and let people post comments instead of having to find an email address of mine.

BTW, Russell and Karl, I posted the links you sent in my del.icio.us links. Thanks again.

Actually, I noticed on /. today that apparently VMware has a free server available now as well as the player (as a beta). Sigh. There just aren't enough hours in the day.

Sunday, April 02, 2006

Attributes vs. Elements

This is a response to Bill Higgins' blog post titled Simplicity for humans, simplicity for programs.

IIRC, when Balaji was designing the new marshaller, I specifically told him that I wanted everything marshalled as subelements and not attributes. Why would I do this? It had nothing to do with readability; I requested this because the marshalling scheme we were replacing was agressively using attributes instead of subelements. Including doing things like marshalling a list of things as a comma-separated string used as an attribute value. That was way wrong. Since it forced folks to have to do further parsing even after parsing up the XML.

The decision on whether to marshall something as an attribute or subelement is usually not too difficult. If the value can be represented as a short-ish string, an attribute works out well. If it's a string and long-ish, using a subelement might be better (and allow you to handle in-line markup on that, or microformats, or ...). If it's data with it's own structure, you'll want to use subelements.

If you are representing a list of things, you really want subelements, although then you have more decisions to make. Should you add a special containing element, like a <ul> element, and render the list items as subelements under that? Or should you not have a containing element, and store each item as a peer of other elements in the same structure. Maybe you want to use a containing element, but that element name is the 'property' name, and the list elements are stored as <li> elements.

In the end, you end up with this weird tension of wanting to use attributes as much as possible, since they are easier for programs to get to, they take up less text, etc. But then you end up having the remainder of your things that you have to render as subelements, for whatever reason. The decision ends up being made strictly on whether XML can actually handle it as an attribute, and that seems wrong.

Thus, I decided that rather than make somewhat arbitrary decisions about whether to use attributes or subelements, and then make people guess at which we'd use when they saw the programs that ate this data, to go with the lowest common denominator; nothing but subelements. Of course, even that's not quite right; we actually do have a small fixed set of things we render as attributes. Very small.

The fact that you have to make decisions like this, even about whether to render something as an element or attribute, speaks to the problems here. XML is great for documents, but what you are really talking about is data. Do we need to describe all our data as documents? If noone is ever going to need it as a document, why go to that trouble? Is XML a solution in search of a problem in this case?

If you don't have a real requirement to provide a document for your data, other formats like JSON might be more appropriate.