Scribble Your Blogs

Scribble is a great language for writing documentation. Now it’s a great language for writing blog posts, too. I’ve just released a tool called Scriblogify that compiles Scribble documents and posts them as blog entries. Scriblogify is a more polished and automated version of the scripts I’ve been using for several months to prepare posts for my own blog.

Multi-file code coverage viewing tool

I'm very pleased to announce the availability of a multi-file code coverage viewer, written by Jonathan Walsh.
Torn between separating your test cases into another file and actually seeing the coverage? Well, go ahead and pull them apart, because the multi-file coverage tool displays coverage information for the files required by the present one, including both percentage covered (on a line-by-line basis) and optionally a list of uncovered lines (no more inching through your code, looking for the red highlighting.
Back End:
One reason I expect this tool to be long-term robust is that it makes absolutely no changes to the back-end; that is, it just uses the existing code coverage framework. The only thing going on here is that the tool provides a way to store, load, and display this information. This means that the tool displays coverage for un-compiled files only. We thought about fiddling with this, but finally decided that the existing behavior was probably about as useful as anything else we'd come up with, and a lot more robust.
URL for docs:
http://planet.racket-lang.org/package-source/jowalsh/code-coverage.plt/1/3/planet-docs/code-coverage/index.html
As you might expect, it's a one-line install:

#lang racket
(require (planet jowalsh/code-coverage))

Please let us know about bugs you discover!

the DrScheme repl isn't the one in Emacs

Dear old Lisper,

You have found drscheme and it is almost like your old Lisp machine. It is easy to program in it, it has things that nobody else has, and we all love parentheses. But after some initial enthusiasm you are wondering why in the world, we decided not to provide commands for sending individual definitions and expressions from the Definitions window to the Interactions window, aka, REPL.

It wasn't an accident. It was by design after some difficult experiences. I am personally a product of the Emacs world that you are describing below, and my advisor Dan Friedman was called the "Lispman" on his door sign at Indiana.

When I first discovered the idea of sending individual expressions and definitions from a buffer to a repl, it was a near-religious revelation to me. I wanted everyone to know this trick and use it. When I started teaching the freshman course at Rice, I told our chairman so and he asked "why". I was shocked, awed, and I failed to explain to him how it mattered. He was a mathematician and I wrote it off. They don't know.

Then I started watching my sophomores and juniors at Rice in lab. Now that was a true disappointment. Few if any used this trick and when they did, they more often tripped up and got the repl into a state where they didn't know what was going on.

In the mid 90s, I wrote some more Little books with Dan, and boy, time and again, I watched him stumble across the state of the repl. I even watched him re-start the repl and load the whole buffer more often than not.

Why? In the presence of macros and higher-order functions and other beasts, it is difficult for masters of the universe with 30 years of experience to keep track of things. What do you think students with 10 or 20 days worth of experience will do? Is it really such a deep principle of computing to create the objects incrementally in the repl as opposed to thinking systematically through the design of a program?

I decided not and asked Robby to make DrScheme's repl transparent. That is, it re-starts the repl and re-loads the buffer every time. I consider this behavior a suitable compromise: have a repl but don't confuse yourself with send-defs and send-exprs. This is especially true in an age when sending an entire buffer takes as much time as sending an individual expression or definition. Soon we'll get "compilation behind your back" so that only the current buffer is re-interpreted. It'll start things even faster.

Even though I had used the incremental mode for more than a decade when I switched from Emacs to DrScheme in 1998, I have hardly ever looked back. I miss a few other things but the incremental repl is one of those rituals old Lispers acquired and never questioned ... but it isn't fundamental and critical to anything. (Note there is no qualifying clauses, no when no if no but. I really mean this last sentence the way I spelled it.)

Simple GnuPLoTting

gnuplot is a very powerful and widely used interactive data plotting program. It can generate two-dimenational and three-dimensional surface plots, either on screen or print to files with a wide-array of supported formats. If you are not familiar with it, you can get an idea of its capabilities here
A new package, gnuplot.plt, is now available on planet which allows you to programmatically interact with gnuplot processes and generate plots on the fly. The package provides a simple interface that abstracts gnuplot's quirky syntax and takes care of data marshalling using temporary files.
Without further ado, here is to every computer scientist's favorite growth curves:

(require (planet vyzo/gnuplot))
(define gplot (gnuplot-spawn))
(define data 
  (gnuplot-data
   (build-list 90
     (lambda (x) 
       (let ((x (add1 (/ x 10.))))
         (list x (log x) (* x (log x)) (expt x 2) (expt 2 x)))))))
;; png output options
(define png '(png enhanced transparent font (str arial)))
;; on screen plot
(gnuplot-set gplot '(title (str "growth curves")))
(gnuplot-plot gplot 
  #:range '(() (1 1000))
  (gnuplot-item data '(using (seq: 1 1) title (str "x") with line))
  (gnuplot-item data '(using (seq: 1 2) title (str "log(x)") with line))
  (gnuplot-item data '(using (seq: 1 3) title (str "xlog(x)") with line))
  (gnuplot-item data '(using (seq: 1 4) title (str "x^2") with line))
  (gnuplot-item data '(using (seq: 1 5) title (str "2^x") with line)))
;; replot to png
(gnuplot-hardcopy gplot "/tmp/grow.png" #:term png)
;; redo with logscale
(gnuplot-set gplot '(logscale y))
(gnuplot-replot gplot)
(gnuplot-hardcopy gplot "/tmp/loggrow.png" #:term png)

Completions in DrScheme (finally)

DrScheme now supports a language- sensitive (but not lexical- scope sensitive) completion feature. Type <menukey>-/ and see what names are available to finish off the word you're typing.

Thanks to Jacob (and do follow that link; we all need a little more love in our lives) and Mike for taking the initiative to actually implement what is probably the most requested feature in DrScheme at the moment.

Adjusting DrScheme's Keybindings

Check out Kyle Smith's blog post on how to change DrScheme's keybindings:

Keybinding 101 in DrScheme

XML Transformation in Scheme

Selenium is a tool for testing web applications, the core of which is a Javascript library that controls a web browser. With the Selenium IDE you can convert your actions in a web browser into tests, and with the Selenium Remote Control you can control a web browser from code. I've recently been working on adding Selenium Remote Control bindings to PLT Scheme, which has resulted in a nice and hopefully instructional demonstration of XML transformation in PLT Scheme

The Selenium Remote Control is controlled by sending simple messages over HTTP. The format of the messages isn't important. What is, is that there are a lot of them, and the API is specified in a file called iedoc.xml that comes with Selenium. The Java/Python/Ruby bindings are generated using XSL. If I was to use XSL I'd have a processing pipeline that uses three languages (XSL, Java, Scheme) which is two more than I'd like. Hence I turned to WebIt!, an XML transformation DSL written in Scheme, to create an all Scheme pipeline. The rest of this post wshows the steps I used to transform the Selenium API into Scheme code using WebIt! I think this is interesting in its own right, but also serves as a nice demonstration of the power of macros, which WebIt! makes extensive use of.

My first step is to get an idea of the structure of the XML. The bits I'm interested in look like this:

<function name="click">
  <param name="locator">an element locator</param>
  <comment>Clicks on a link, button, checkbox or radio button.
  If the click action causes a new page to load (like a link usually
  does), call waitForPageToLoad.</comment>
</function>

Let's read in the XML file and extract all the function elements. For this I'll use SSAX and SXPath:

(require
 (planet "ssax.ss" ("lizorkin" "ssax.plt" 1))
 (only (planet "sxml.ss" ("lizorkin" "sxml.plt" 1)) sxpath))

(define api
  (with-input-from-file "iedoc.xml"
    (lambda () (ssax:xml->sxml (current-input-port) '()))))

(define functions
  ((sxpath '(// function)) api))

Ok, so we have all the functions. Now let's parse them into a more useful datastructure. Here's my first attempt:

(require (planet "xml.ss" ("jim" "webit.plt" 1 5)))

;; struct function : string (listof string)
(define-struct function (name params))

;; parse-function : sxml -> function
(define (parse-function fn)
  (xml-match fn
    [(function name: ,name
               (param name: ,param-name ,desc) ...
               (comment ,_ ...))
     (make-function name (list param-name ...))]))

(map parse-function functions)

The xml-match macro is a pattern matcher for SXML. You specify the “shape” of the SXML, and if the input matches the pattern the following expressions are evaluated:

(xml-match value
  [(pattern expression ...)]...)

The simplified form of a pattern is:

• (element ...) matches an element with the given name.
• name: value matches an attribute with the given name and value.
• ,binding binds the value of binding to the given name in the scope of the following expressions.
• ... matches zero or more of the preceeding patterns.

In our example the pattern is:

     (function name: ,name
               (param name: ,param-name ,desc) ...
               (comment ,_ ...))

So we're looking for an element called function with an attribute called name the value of which is bound to name. Then follows zero or more param elements, with attribute name, the value of which is bound to param-name. Finally we expect a comment element which can contain any amount of data. The use of _ as the binding name is a common convention to indicate data we don't care about but must still match to make our pattern complete.

I run the code in DrScheme and see the result:

xml-match: no matching clause found

Oops. So our pattern isn't complete. We've also seen one flaw of WebIt!: it doesn't give very good error messages. However we can easily fix this by adding a catch all pattern that raises an error telling us what we failed to match. The code follows. Notice that I've also added pretty printing to make the unmatched SXML easier to read.

(require (lib "pretty.ss"))

;; parse-function : sxml -> function
(define (parse-function fn)
  (xml-match fn
    [(function name: ,name
               (param name: ,param-name ,desc) ...
               (comment ,_ ...))
     (make-function name (list param-name ...))]
    [,err (let ([op (open-output-string)])
            (pretty-print err op)
            (error (format "Didn't match ~n~a~n" (get-output-string op))))]))

Run this code and you'll see the error occurs as we don't allow the description to contain more than one element. This is easily fixed by extending the pattern to ,desc .... The next error is more interesting. The function element contains a return element. The WebIt! pattern language doesn't allows us to express optional patterns, so we have to duplicate our pattern and include the case of return. This also requires we extend the defintion of the function structure.

;; struct function : string string (listof string)
(define-struct function (name return params))

;; parse-function : sxml -> function
(define (parse-function fn)
  (xml-match fn
    [(function name: ,name
               (param name: ,param-name ,desc ...) ...
               (comment ,_ ...))
     (make-function name "void" (list param-name ...))]
    [(function name: ,name
               (return type: ,type ,return-desc ...)
               (param name: ,param-name ,desc ...) ...
               (comment ,_ ...))
     (make-function name type (list param-name ...))]
    [,err (let ([op (open-output-string)])
            (pretty-print err op)
            (error (format "Didn't match ~n~a~n" (get-output-string op))))]))

This works! This is as far as I want to go in this article. We've seen how we can use SSAX. SXPath, and WebIt! to create XML transforms in pure Scheme. There is a lot more to all of these packages but what we've used is sufficient for many uses. The rest of the code to create Scheme from the API is quite straightforward and specific to Selenium. If you're curious read the source of the Selenium PLaneT package, which will be released soon.

This post also appears on Untyping