Links

pmuellr is Patrick Mueller, Senior Node Engineer at NodeSource.

other pmuellr thangs: home page, twitter, flickr, github

Friday, December 17, 2010

weinre at GitHub

I finally pushed my weinre project up to GitHub last night:

weinre - web inspector remote - pronounced "winery"
http://pmuellr.github.com/weinre/

It's a debugger for mobile web apps. Uses the Web Inspector debug front-end from WebKit (implemented in HTML/CSS/JS), plugged up to some plain old JavaScript running in your web page. There's an HTTP server in there somewhere.

It's early days - it may not be the easiest thing to set up and run - and there's a lot of function not implemented (showing all the CSS style info, for instance). But it's something.

I intend to continue enhancing it over the next while. First order of business will be to get it back, function-wise, to where my earlier prototype was, as shown in this video. I've been spending more time on infrastructure stuff than adding debug function lately - it'll be fun to dive back into the innards of Web Inspector again.

My intention is for this work to eventually be folded into the PhoneGap project. PhoneGap (wisely IMO) requires a Contributor Agreement in place before accepting contributions, so I won't be able to accept any contributions into my repo unless you've signed up.

If you have questions, feature requests, etc, create an issue at GitHub. I also hang out on #phonegap on freenode during business hours (US east).

Tasty Innards Too!

Besides this project's subject matter being a fun thing to work on (building a working debugger), I've had a lot of fun building and using smaller thingees. If I can get my blogging going again, hopefully I'll write more about some of these goodies, soon:

  • All my JavaScript code is structured as CommonJS modules, using modjewel.

  • All my JavaScript is written in a pidgin dialect of JavaScript called scoop. Basically just a lexically simpler way to define classes, methods, etc in JavaScript (if you didn't know already, I'm an old Smalltalk dude). Here's an example - the implementation of the Console class used by the debug target.

  • The interface between the large grained components of weinre is specified in a subset-but-extended bastardization of WebIDL. The Web Inspector code already does this itself (because WebKit in general does this also), and it worked out well for me. I wrote a WebIDL-to-JSON compiler, and the runtime makes use of the JSON to create proxies, etc. Lots more interesting stuff can be done here, I'm sure.

Tuesday, December 14, 2010

tech books on the Kindle

Three images from David Flanagan's JQuery Pocket Reference, as they look on my 3rd generation (latest) 6" (not the big one) Kindle.

The first image below is from the PDF version of the book.

The font size is a little too small for me, and I would assume, most people. It also wastes a lot of space on the left and right margins in general, on the indentation of the code sample, and the footer information.

Note that PDFs are not "resizable" in the same way a web page is; you can't make the font larger. What you see is what you get.

The next image below is from the Mobi version of the book. Mobi is a publishing format that has been used by various reader software over the years, and is the "native" format for the Kindle.

What I'm showing here is the text at the smallest font size supported. I typically read at one font size up from that. There are 8 font sizes available when you're reading Mobi files, along with being able to tweak the line spacing a bit, etc.

I'm showing the smallest font size because it shows one of the biggest problems with using Mobi for tech documents - the code sample is too big for this font size (the smallest!) and it wraps. It just looks terrible.

I have some experience with the Mobi format, converting HTML documents to Mobi using Calibre. It appears the Mobi format is pretty limited. I've tried all sorts of styling of <pre> elements, and the best you can do is force some vertical whitespace (not the default!) in front of your code samples. It's just awful.

What's a boy to do? Turns out O'Reilly also offers an ePub version of their eBooks. You can easily turn your ePub into a Mobi with Calibre, but you're not going to be able to do much better than the image above.

So I did something different:

  • unzip the ePub - ePub files are zip files with an .epub file extension

  • concatenate all the .html files in the epub together, into one file - ePub files are XHTML files for their main text content

    cat ch*.html > combined.html
    
  • remove the <head> sections with a find/replace regex pattern, etc

  • add a simple <style> section:

        h1.title {
            page-break-before:  always;
        }
    
        body {
            font-size:          200%;
            font-family:        Georgia;
        }
    
        pre {
            font-size:          70%;
            background-color:   #EEE;
            padding:            0.5em;
            overflow:           hidden;
            border-radius:      0.5em;
            -moz-border-radius: 0.5em;
        }
    
        .sidebar {
            padding:            0.0em 0.5em;
            overflow:           hidden;
            border-radius:      0.5em;
            -moz-border-radius: 0.5em;
            background-color:   #EEE;
        }
    
        .sidebar p.title {
            font-weight:        bold;
        }
    
  • bring the resulting file up in FireFox

  • print, but save to PDF instead of actually print (I'm on a Mac)

  • copy the PDF to the Kindle

Here's the result. Not perfect, but much better than the other two files. The font is larger, and the code sample is readable and well separated from the other text.

Note: FireFox is not my day-to-day web browser, but is the only browser that produces usable PDF files for files like this. PDFs produced from both Safari and Chrome only last about 30 pages or so before the text disappears, when viewed on the Kindle. Who knows.

Update on 2010/12/14 at 1:20pm - the images aren't working from flickr, so put on dropbox

Friday, June 04, 2010

bind() considered harmful

A recent passive-aggressive twitter message from me:

Oh yeah. Keep on bind()'ing folks, because bind() rocks. http://bit.ly/aPyfuV

In case you can't tell, the message is facetious. bind() does not actually rock.

what is bind()?

bind() is a JavaScript function implemented by many JavaScript libraries and included in ECMAScript 5. If you aren't already familiar with bind(), this post isn't going to make much sense, but if you're curious anyway, see Prototype's documentation for their bind().

Let's look at what bind() is actually doing:

  • associates an object to use as this (aka the receiver) when the function is invoked
  • associates additional parameters to be passed to the function when the function is invoked (aka currying).

bind() does this by returning a new function, which when invoked, arranges for the object you want as the receiver and the additional parameters to be passed to the original function. It hides these values in the newly created function, associated with the closure the function was returned from.

bind() is frequently used for callbacks and event handlers, which typically allow you to pass a function as the callback, but don't provide a way to specify the receiver or additional parameters for the actual call to the callback.

why does bind() not rock?

  • gets in the way of debugging

    Take a look at the bug I linked to in the Twitter message above: Bug 40080: Web Inspector: better Function.prototype.bind for the internal code

    The bug concerns providing a better debug story around functions that have been bind()'d. Because the current story isn't very pretty. When you stumble upon a bind()'d function in the debugger, you see the source for the bind() function itself, which isn't what you want to see. In addition the resulting bound function is usually anonymous, which means your stack traces, profile reports, etc, will be filled with (anonymous function) entries.

  • garbage

    Calling bind() creates a new function object every time it's called. Not just a string, or empty object, or array. A function. Which I'm guessing is more expensive than simpler objects. [yes, I should do some measurements.] A closure is also created and associated with the function (which is where the receiver and curried arguments are stored). More garbage.

    Now imagine that, for whatever reason, you need to add and then remove a callback frequently, over and over again, for some reason. Or set a timer over and over again. And you're using bind(). Think of the garbage you're creating.

  • Alex Russell posted some thoughts on bind() in a post to the es-discuss mailing list (item 3).

    So why does this [using bind()] suck? Two reasons: it's long-ish to type, and it doesn't do what the dot operator does -- i.e., return the same function object every time.

    It is longer, and therefore yuckier. It's also often not DRY (obj referenced twice):

        setTimeout( obj.method.bind(obj, "John"), 100 );
    

    The second point is explained by Alex in his post:

    Many functions, both in the DOM and in the library, accept functions as arguments. ES5 provides a bind() method that can ease the use of these functions when the passed method comes from an object. This is, however, inconvenient. E.g.:
       node.addEventListener("click", obj.method.bind(obj, ...));
    
    Also, insufficient. Event listeners can't be detached when using Function.prototype.bind:
       // doesn't do what it looks like it should
       node.removeEventListener("click", obj.method.bind(obj, ...));
    

    So the trick here, if you want to use removeEventListener(), is that you have to store the result of a single call to bind() and then use that value in subsequent addEventListener() / removeEventListener() paired calls. Alex's post suggests a new language feature to work around this (btw, I'm not in favor of his proposed language feature).

why are we in a bind with bind() today?

It's pretty obvious to see how we got to the point where you need to use bind() in your code today.

Historically, JavaScript was a glue language that let you do a light amount of programmatic processing against stuff in your page. When specifying a callback/listener, you didn't have to worry so much about the receiver of the function you passed in; you were probably using global variables instead of creating your own little objects.

And so the places that take callbacks, like setTimeout(), onload handlers, etc, didn't really have a need for you to specify the receiver of of the callback when it was invoked. The receiver was always ... well, whatever it was for your callback.

Fast-forward a decade, and now we have people building huge systems out of JavaScript, using some sort of "class" story, or living the hippy prototype lifestyle, or who knows what kids are doing today. In any case, there's often an "object" in the picture, and you'd often like to arrange to have that object be the receiver of the callback. Quite often, you'd like for the callback function to be a method of an object, and have the receiver of the callback be that object.

The problem is, there's no where to put the receiver; all the pre-existing callback patterns just allowed the use of a function parameter. The trick with bind() is that it attaches the receiver, and possibly curry'd arguments, to an invisible bag wrapped around a newly created function which is a delegated version of your callback function. Nature will find a way.

how can we fix the evils of bind()?

For me, the root of the problem is that we're passing the method receiver in a secondary channel, the bound function. So, stop doing that. Pass it explicitly.

Let's play with changing the addEventListener() function to accommodate a new receiver parameter. Here's the current function signature:

    target.addEventListener(type, listener, useCapture)

We can add the receiver parameter to the end of the function:

    target.addEventListener(type, listener, useCapture, receiver)

or we could allow listener and receiver to be combined together in an array and used where the existing listener value is today:

    target.addEventListener(type, [receiver, listener], useCapture)

This second flavor tastes better to me.

what does the fix smell like?

Let's compare the code. For the examples below, obj is the receiver of the callback, callback() is a method available on the obj object.

Using ECMAScript 5's bind() method:

    node.addEventListener("click", obj.callback.bind(obj))

Here's the four-arg addEventListener():

    node.addEventListener("click", obj.callback, false, obj)

Here's the two-element-array-listener addEventListener():

    node.addEventListener("click", [obj, obj.callback])

These examples could be made even DRYer, if instead of passing a function reference, you pass a string, which will be used as a property name to obtain the function from the receiver object:

    node.addEventListener("click", "callback", false, obj)
    node.addEventListener("click", [obj, "callback"])

In terms of being able to handle the removeEventListener() case as well, when using the ECMAScript 5 version of bind() you would have to arrange to store a copy of the bound function, so you can send the exact same function both addEventListener() and removeEventListener(). My proposed versions could do a compare against the parameters or array elements, allowing you to use the exact same parameters on addEventListener() and removeEventListener().

In other words, here's how you do it in ECMAScript 5:

    var boundFunction = obj.callback.bind(obj)
    
    node.addEventListener("click", boundFunction)
    ...
    node.removeEventListener("click", boundFunction)

And here's how you do it with my proposal:

    node.addEventListener("click", [obj, "callback"])
    ...
    node.removeEventListener("click", [obj, "callback"])

This invocation pattern works the same for the other form of call that I proposed.

Curried arguments can be handled the same way the receiver parameter is handled; passed as additional arguments to addEventListener() (not needed for removeEventListener()?), or an additional element in the array where the listener argument was previously used. One simplification would be to allow a single curried argument - other callback systems typically refer to this as userData or clientData - rather than deal with a variadic list. It's simple enough to combine multiple curried arguments into an object or array for use as a userData argument.

actually, bind() isn't always evil

Although I've spent this entire post complaining about bind(), I will acknowledge it's power and usefulness. Particularly in meta-programming and function programming.

My primary complaint is having to use bind() is something pedestrian as callbacks. That's too much.

the obligatory Smalltalk reference

My views on this subject are biased by my exposure to OTI/IBM Smalltalk. Read up on the Callbacks section on page 150 of "IBM Smalltalk: Programmer's Reference".

You'll note that my suggestion here is no different than the addCallback:receiver:selector:clientData: method described in that manual, just a shorter name.

Wednesday, June 02, 2010

coopitition

A neat thing happened last month. I entered a feature request against the Web Inspector debugger shipped with WebKit, to get some new function added. Two weeks later, that function was implemented in the FireBug debugger for FireFox. Coincidence?

The feature request was this one: Bug 38664: Web Inspector: add a "table" method to console, to allow output of tabular data. I got the idea for this from looking at another Web Inspector bug: Bug 30897: Web Inspector: Support Debugging via HTTP Headers like FirePHP and FireLogger. Turns out that FirePHP (another interesting project) has had support for tabular console output for a while. I immediately felt left out.

So how did the FireBug folks beat the Web Inspector folks (me included) to the implementation? Turns out that IBM employs folks who work on both Web Inspector and FireBug. My primary contact for FireBug stuff in IBM is John J Barton, and I thought this new function would be interesting enough they'd want it too. I guess I was right. John and I trade notes on stuff like this from time to time.

Once Jan Odvarko (aka "Honza") posted his blog entry Tabular logs in Firebug, my interest in this bug grew, for some reason :-) So, had some conversation with Jan, on the bug, last Friday. It was great for Jan to provide additional feedback in WebKit's Bugzilla. And then I was totally surprised yesterday morning to find that a bunch of additional great ideas around the feature request got posted back to the bug over the weekend, from Jan and other Web Inspector developers.

I decided to try to summarize where we were, provide some examples, and ended up writing a little console.table() simulator to play with, all of which is currently available in this file: console.table() proposal. Please post comments on that file back to the bug, for now.

I love friendly competition!

Wednesday, April 28, 2010

the weinre experiment - WEb INspector REmote

For the last few months, one of the things I've been working on is an experiment called weinre, which stands for WEb INspector REmote. The basic ideas are:

  • reuse the existing Web Inspector UI code, implemented in HTML, CSS, and JS
  • get that Web Inspector code to run as a plain old web page
  • have that Web Inspector code debug a web page running in another browser window, perhaps on another machine

You might ask - "Why?" The answer is, largely: mobile. How am I supposed to debug mobile web apps on an iPhone, Android device, or similar? Today the answer is: painfully. Tomorrow the answer should be: via a nice debug user interface like Web Inspector.

The source and an executable demo are available on WebKit's wiki, currently at the bottom of the Web Inspector page:

I have an introductory write-up of the experiment at my web site, and it's also included with the source and executable demo:

If you are truly lazy and don't want to bother with those, but can afford 3 minutes for a demo movie, you can watch it on YouTube. Best viewed in 720p (HD) in full screen mode, if your browser supports that.

Let me emphasize that this is an experiment. Although parts of it do work, as shown in the movie, it's a bit unwieldy and insecure to use as a production tool. I was at a good place to stop, and so thought I'd archive the wad up for other people to experiment with.

So, what's next?

  • Security, as noted in the write-up, is a big concern of mine. Must do something there.

  • Remove the requirement for the separate agent/server. Meaning that one of the debug client or the debug target should act as a server to it's peer. Or perhaps both debug client and debug target should be able to act as the server. Or use some other whacky IPC mechanism.

  • Formalize some structure around the messages sent between the target and client. Today the messages sent between the debug target and debug client are very specific to the Web Inspector implementation. It would be nice to have something less specific. One of them things called an A-P-I.

  • Not sure where my time is going to be spent over the next year, but it might not be here, so the answer to "What's next?" might be: "nothing, from me". But the tea leaves seem to be reading: "remote web debug".

Friday, February 26, 2010

prior art

A few caveats before starting:

  • I am not a lawyer, just an old country programmer, currently working at IBM.
  • This blog post is my own and does not necessarily represent IBM’s positions, strategies or opinions.
  • The patent infringement suit I mention below does not involve IBM as a litigant. It does involve me as a fact witness.
  • No need to go into details on the patent infringement suit, as they are irrelevant to this blog post.

Tim Bray recently wrote a blog post, "Giving Up On Patents", providing his opinion on the state of patents and linking to a number of interesting patent-related texts on the web. My thoughts are currently roughly aligned with Tim's, so there's no need to restate them.

Since I was recently involved as a fact witness in a patent infringement suit, I thought I'd provide some tips if the following are true for you:

  • You believe that there are a lot of patents granted for which there already exists prior art, and so those patents really shouldn't be granted in the first place.

  • You make things.

The tips:

  • Document the crap out of the things that you make.

  • Put your documentation on the web, and make sure search engines can find it 20 years from now.

For the suit I was involved with, an attorney of one of the defendants did a Google search on a term relevant to the suit, and found a paper I had written 13 years ago, on my personal web site. The paper they found lead them to believe I had information relevant to the suit; they thought I had evidence of prior art. Eventually, a lot more documentation was excavated from various people and places.

I found it interesting that a couple of movies were added to the pile of evidence I was asked about during my deposition, all of which appeared to be digitized versions of tape. They were all also around 13 years old. I suspect video will become even more popular as evidence in the future, given the ease in creating and disseminating it, and the amount of information it provides. Lawyers love information. Think screencasts, or perhaps ASCIIcasts are more relevant for the thing you made.

In case you're morbidly curious, I don't believe the statements I provided during my deposition actually helped the defendant in any way. Based on the questions I was asked, I have my guesses as to whether I helped anyone, but I have no way of knowing. I didn't study the patent in question, so I don't know what claims the plaintiff is making, nor do I know how the defense is planning to defend against those claims. All that is irrelevant for a fact witness; I was there to objectively answer questions that were asked of me. Just the facts, ma'am.

A couple of slightly humorous notes:

  • During the deposition, the court's Windows machine could not be coaxed to play the videos (missing codecs). One of the younger attorneys saved the day with his MacBook.

  • Making some small talk (heh) with my counsel during a break, I mentioned that I might claim "software patents are against my religion" if I was ever asked to do something I really didn't want to do. He had a good laugh, and said he'd never heard that before. Note taken.

  • The paper that the attorney found on the web had a self-referential link in it. I always do that, never know what might happen to an HTML file. The link, however, was to a web site long since gone, which today renders as a parked 404 page somewhere. Of course, the paper had moved somewhere else, and of course, Google knew where it was. But it was still funny to see that old link.

  • Being a fact witness doesn't pay well. I received a $55 check for my time; the deposition was an all-day event.

Tuesday, February 09, 2010

modjewel 0.2.0 - a CommonJS require() function for the browser

I decided to get off my butt and bring my "for the browser" CommonJS require() implementation at http://github.com/pmuellr/modjewel into the 20-teens.

some background

Just over a year ago the "ServerJS" group was formed based on the "What Server Side JavaScript needs" blog post from Kevin Dangoor. This effort was renamed "CommonJS" later in the year, and lives on the web here: http://commonjs.org/. Kris Kowal has a nice write up on CommonJS at Ars Technica in the article "CommonJS effort sets JavaScript on path for world domination", if you don't know anything about it. Note: I'm a sucker for software-based world domination.

One of the first items looked at by the CommonJS community was a "module" system, centered around a function named require(). More information on modules and require() is available here: http://wiki.commonjs.org/wiki/Modules/1.1. While most of the folks in the group were playing with server-side JavaScript, I was interested in the browser, so cooked up a crude version of require() in a github project just over a year ago; the crufty old hack is here.

I figured someone else would do a proper version, but haven't really come across anything besides David Flanagan's efforts, documented in his blog posts "CommonJS Modules implementation" and "A module loader with simple dependency management". I have to admit I don't watch the CommonJS stuff all that closely, I probably missed some other implementations.

David's code doesn't have any kind of a license associated with it, so I didn't really feel it was appropriate to use it for a real project. And there were some things I wanted to modify. So, at the very least David shamed me into sprucing up modjewel and I will admit to stealing bits of his design as well.

One thing I learned from David's blog post was that compliance tests exist. Loves having tests. They currently live here: http://github.com/kriskowal/commonjs. Note that David's blog post points to an older version of the tests.

Onto the goodies in the newly improved modjewel ...

nice ui for the compliance tests

As I mentioned above, there's a set of test cases for the require() function. Problem is, they're built for command-line invocation; remember, this CommonJS stuff is more server-side than client-side JavaScript. I was recently pawing through the source for the SunSpider JavaScript benchmarks, and realized I could use the same "run the tests serially in IFrames" technique here, so I lifted that design. I now have a single page that will load and run all the tests serially, and produce a green/red report card when done. It's not perfect, but it's very handy.

One weird thing to note was that I couldn't find any kind of license on the tests; I ended up importing them directly into my project, which is kind of silly. I should probably run them right out of the github repo.

support for module.setExports()

The existing facilities with modules make it easy to provide access to a number of objects within a particular module. However, for those times you only have a single object in a module, things get verbose. Here's an example: say you want to build and use a module called WebDav (so the file is named WebDav.js) which exports a single function, WebDav.

In the file WebDav.js, you would define the WebDav class as:

   exports.WebDav = function WebDav() {
       // the function is 'named' for debug purposes
   ...
   }

You can then use the WebDav class in another file as:

   var WebDav = require("WebDav").WebDav

All the RY of Java. The "WebDav" passed to require() specifies the module, and the next WebDav selects the function as a property of the exports object the module populated.

Alternatively, with the module.setExports() support you can define the class in WebDav.js as:

   module.setExports(function WebDav() {
   ...
   })

and then use it in another file as:

   var WebDav = require("WebDav")

Better! Very convenient if you have a 1-1 correspondence between classes and modules.

module.setExports() isn't official yet; see the wiki page for more information: http://wiki.commonjs.org/wiki/Modules/SetExports.

preloading modules via <script src="">

The way modjewel's version of require() works is like every other browser-based one: it use synchronous XHR. Which is ok for development-time, but completely unacceptable at deployment-time. So they say. Flanagan came up a preload story for his require2.js file, but the mechanism seemed a little crude.

Instead, I created a command-line utility which will convert a module into a file which can be embedded with <script src="">. So then the story would be that you run this utility as part of a build step, perhaps minize or whatever you kids do to your JavaScript afterwards. In the end, you have a "plain old" JavaScript file to "include". All the utility really does is wrap your code in a function. When this code is executed, it registers the module and the function, such that the first time require() is called to load the module, the function is executed and the module is loaded.

A nice aspect of this is that the order in which the <script src=""> are placed doesn't need to be in some kind of special pre-req order. Including the modules doesn't really load them, it just pre-loads them.

Anyway, it's one answer to the "you can't use synchronous XHR!" argument.

portability

I've run the tests successfully on Mac Chrome, Mac Safari, Mac Opera 10, the IPhone Simulator, Windows Chrome, and Windows FireFox 3.0. IE 7.0 was a no-go.

There is an issue with Chrome, where some tests fail when running against a file:// URL. See issue 1 for more info.

There are some jiggly issues with Windows - of course those being path separator issues (\ instead of /) and new-line issues (CR LF instead of LF). Should be easy to fix. Not sure what to do about IE 7, do I use FireBug Lite to debug that?

Monday, February 01, 2010

remote web inspector - baby steps

Last week, I did a little work getting a remote debug story between an iPhone and WebKit's Web Inspector limping. Barely limping. More like crawling. Anyhoo, the write-up is here:

http://muellerware.org/papers/remote-web-inspector-baby-steps.html