There's been a bit of controversy over Mike Pence's article "Heresy and turtles (all the way down) with Avi Bryant"; it's an interesting read, if obviously, and seemingly intentionally, controversial.
There is one bit that particular cries out to me, and that's the notion of Smalltalk being implemented as "turtles-all-the-way-down". Meaning, Smalltalk 'libraries', like collections, http client access, numbers (heh) are implemented in ... Smalltalk. With a small core of 'natives' that do the lowest-level stuff like opening files and writing to sockets. Alternatively, if you look at a language like PHP, you realize that none of the 'base' library is implemented in PHP, it's implemented in C. Notice that these two languages are at opposite ends of the spectrum here, compared to other languages like Ruby and Python, where there's more C code than in Smalltalk, but less than in PHP (relatively, compared to libraries implemented in the target language). Meaning, a lot of the languages we use, have a lot of their base library code implemented in C. And not just any old C, but C code wrapped in layers of language- and interpreter-specific wrappers. Meaning, I can't share that code between languages. Ruby can't use 'extension libraries' built for Python (and vice versa). It gets worse. JRuby can't use 'extension libraries' built for Ruby (and vice versa).
Something's clearly a bit out of whack here. It shouldn't be this hard. There's so much other work involved in getting new languages, or new implementations of existing languages up and going, not to mention making sure you have good editors, debuggers, etc. This is clearly one area that we can make easier for language implementors, isn't it?
In VisualAge Smalltalk, we had both a 'native' interface, with icky wrappers you had to use around your C code, but we also had a great little framework called PlatformFunction. A way to call out directly to C code. We also had a framework to allow you to deal with native pointers called OSObject. With these facilities, in the 5 or so years I was shoulder deep in Smalltalk, I think I only ever wrote one real 'native'; I was able to use call outs to C code for everything else I needed. And I used this a lot. We also had a way to generate a pointer to a C function that hooked into an arbitrary method, so you could call in to Smalltalk from C, for callback purposes.
I've seen other libraries in other languages do this as well; usually referred to as ffi - foreign function interface. The CLR provides for this as well, via P/Invoke. It's a nice capability that every language environment should provide.
The downside of this is that you probably can't do anything with 'objects' using these interfaces. You're dealing with low-level C goop. Sometimes you really want to be dealing with objects.
So here's an interesting thought exercise. Would it be possible to create a portable 'native' interface, like Java's JNI, which can deal with 'objects', that could be used by multiple languages? So that I could compile an extension library as a binary, native shared library that multiple languages could make use of. I call out JNI specifically both because I'm intimately familiar with it, and it's actually a pretty nice API. For a number of reasons:
JNI defines all the 'functions' used to talk to the runtime that you can use from your natives, as function pointers in a struct that gets passed to your 'native' functions. Since they're function pointers, you don't need to link against a library; the functions are resolved at runtime, not link time. This means I can compile a Java native library once, and use it on any VM that supports the same level of JNI.
JNI provides pretty nice encapsulation from the VM implementation. Instead of getting pointers to internal VM structures and mucking with stuff, you have function calls to do your work. Instead of dealing with garbage collection bits and pieces (like reference counts), you use APIs to create/destroy references that the underlying VM will use to deal with the garbage collection bits and pieces.
JNI has been protected very well against version to version changes, so that you have a very good expectation that a native library you compiled for version 1.x of Java will work with version > 1.x of Java.
What would it mean to do something like this to support multiple languages? There are obviously too many semantic differences between all the different languages we use to hope to be able to have a complete, universal 'extension' interface, but there's also a lot of commonality with these languages. It's not a simple problem to solve. But it's time to start thinking about it. In fact, it seems a bit embarrassing that we've not yet started thinking about it.
No comments:
Post a Comment