pmuellr is Patrick Mueller

other pmuellr thangs: home page, twitter, flickr, github

Saturday, April 21, 2007

Embedded Gnome

The announcement last week of the GNOME Mobile & Embedded Initiative was pretty interesting. I hope this breathes some fresh air into the embedded space.

It's been a while since I worked in the embedded software development arena, but boy was it fun. I highly recommend, if you ever get the chance, of doing some software development in the embedded world, if all you've ever done is desktop and server software development. You'll never look at multi-megabyte jar files of classes in the same way.

I still have the first embedded board I got code running on sitting on my desk; a 403GC PowerPC board. 25 or 33Mhz, can't remember how much RAM I had on it, using whatever version of QNX was available at the time. I finangled an expansion board from one of the embedded hardware guys that had an 8 segment LED so that I could actually output something somewhere besides a telnet session. This would have been ~1997, so I had the obligatory live stock ticker running on it. In Smalltalk, of course.

I'm guessing that Apple's re-entrance into the embedded space with the iPhone has prompted this action, to some extent. Embedded devices are sexy again. I've even been using my old Palm Tungsten-C now and again, if I think I'm going to be out somewhere with free wifi, so I can Twitter.

And Mike Wilson pointed out that Opera is running on the Palm, and is interestingly enough implemented on the J9 Palm Java VM.

Down at the bottom of the GNOME Mobile & Embedded Initiative announcement page, under "Technologies Under Consideration", is listed "Java ME" (Micro Edition). Hmm. Wonder what they're thinking. My wish would be for CLDC, which is the smallest useful Java class library available for the J2ME space, and then add SWT for the UI. Not sure if the eSWT libraries have GTK bindings, but I'm sure that's possible. I certainly hope they don't select just MIDP, since it's so weird and low-functional, and certainly hope they don't select Personal Profile or Personal Basis as the rich GUI, because that's just a step into the past (AWT).

Glad to see things are 'opening up' a bit, in any case.

Friday, April 20, 2007

turtles all the way down

There's been a bit of controversy over Mike Pence's article "Heresy and turtles (all the way down) with Avi Bryant"; it's an interesting read, if obviously, and seemingly intentionally, controversial.

There is one bit that particular cries out to me, and that's the notion of Smalltalk being implemented as "turtles-all-the-way-down". Meaning, Smalltalk 'libraries', like collections, http client access, numbers (heh) are implemented in ... Smalltalk. With a small core of 'natives' that do the lowest-level stuff like opening files and writing to sockets. Alternatively, if you look at a language like PHP, you realize that none of the 'base' library is implemented in PHP, it's implemented in C. Notice that these two languages are at opposite ends of the spectrum here, compared to other languages like Ruby and Python, where there's more C code than in Smalltalk, but less than in PHP (relatively, compared to libraries implemented in the target language). Meaning, a lot of the languages we use, have a lot of their base library code implemented in C. And not just any old C, but C code wrapped in layers of language- and interpreter-specific wrappers. Meaning, I can't share that code between languages. Ruby can't use 'extension libraries' built for Python (and vice versa). It gets worse. JRuby can't use 'extension libraries' built for Ruby (and vice versa).

Something's clearly a bit out of whack here. It shouldn't be this hard. There's so much other work involved in getting new languages, or new implementations of existing languages up and going, not to mention making sure you have good editors, debuggers, etc. This is clearly one area that we can make easier for language implementors, isn't it?

In VisualAge Smalltalk, we had both a 'native' interface, with icky wrappers you had to use around your C code, but we also had a great little framework called PlatformFunction. A way to call out directly to C code. We also had a framework to allow you to deal with native pointers called OSObject. With these facilities, in the 5 or so years I was shoulder deep in Smalltalk, I think I only ever wrote one real 'native'; I was able to use call outs to C code for everything else I needed. And I used this a lot. We also had a way to generate a pointer to a C function that hooked into an arbitrary method, so you could call in to Smalltalk from C, for callback purposes.

I've seen other libraries in other languages do this as well; usually referred to as ffi - foreign function interface. The CLR provides for this as well, via P/Invoke. It's a nice capability that every language environment should provide.

The downside of this is that you probably can't do anything with 'objects' using these interfaces. You're dealing with low-level C goop. Sometimes you really want to be dealing with objects.

So here's an interesting thought exercise. Would it be possible to create a portable 'native' interface, like Java's JNI, which can deal with 'objects', that could be used by multiple languages? So that I could compile an extension library as a binary, native shared library that multiple languages could make use of. I call out JNI specifically both because I'm intimately familiar with it, and it's actually a pretty nice API. For a number of reasons:

  • JNI defines all the 'functions' used to talk to the runtime that you can use from your natives, as function pointers in a struct that gets passed to your 'native' functions. Since they're function pointers, you don't need to link against a library; the functions are resolved at runtime, not link time. This means I can compile a Java native library once, and use it on any VM that supports the same level of JNI.

  • JNI provides pretty nice encapsulation from the VM implementation. Instead of getting pointers to internal VM structures and mucking with stuff, you have function calls to do your work. Instead of dealing with garbage collection bits and pieces (like reference counts), you use APIs to create/destroy references that the underlying VM will use to deal with the garbage collection bits and pieces.

  • JNI has been protected very well against version to version changes, so that you have a very good expectation that a native library you compiled for version 1.x of Java will work with version > 1.x of Java.

What would it mean to do something like this to support multiple languages? There are obviously too many semantic differences between all the different languages we use to hope to be able to have a complete, universal 'extension' interface, but there's also a lot of commonality with these languages. It's not a simple problem to solve. But it's time to start thinking about it. In fact, it seems a bit embarrassing that we've not yet started thinking about it.

Thursday, April 19, 2007

setting up Mercurial on TextDrive with ssh

I've been thinking about setting up a simpler, change-set based source code repository for my little personal projects for a while now. For the past six months or so I've been using SVN on my TextDrive account but ... looking for something different.

I ran across Mercurial (hg) at least twice now recently, so that speaks to me "check it out". Getting it set up on my local box was straightforward, and of course the next step is getting it set up on your server. I host at TextDrive. And Bill de hÓra recently posted some instructions on how to do just that. Kind of.

The problem with Bill's instructions are that you're setting it up to run using Basic Auth but using http instead of https. Bill points out himself, that "This isn't secure". Using https with TextDrive is possible, but at times confusing. Luckily, there is a fairly straight-forward way to get to a secure solution.

The basic idea is to use the ssh instead of http for when accessing your hg repositories programmatically. Instructions on using ssh with hg are available in the hg(1) man page.

In a nutshell, follow Bill's instructions, with a few slight changes, and it just works, assuming you have ssh set up in the first place.

If this doesn't work, ping me; I perhaps forgot a step.

I also created a ~.hgrc file on my client box, with the following contents. The ssh setting causes ssh to use compression when transfering the data/files.

    username = your name <yourname@yourmail>
    ssh = ssh -C

The only thing I don't like about the resulting set up is having to update the hgweb.config file on the server whenever you create a new repository. Though I suppose it's nice that you can selectively make repositories easily publicly accessible or not this way. I don't think I'd rely on this trick to hide something.

I copied Bill's instructions, and then augmented them with the changes I made, which you can find here. I would have included them in this blog entry, but Roller decided to strip all the nice formatting I did out my post. And that's just unacceptable, given the stupid amount of time I spent adding it in, in the first place.