Don't say "abstract" (instead say "general")

The word "abstract" is common in computer science. An abstract thing is one where some part of the whole is unspecified. For instance, the expression "3*x + 3" is an abstraction of the expression "3*4+3", because the "x" is unspecified. Likewise, a function is an abstraction over some set of values, supplied when the function is called.
The word "general" is not at all common in computer science. In non-computer-science use, the word "general" is used to describe things that may be applied to more than one thing or situation. For instance, a "more general solution" is one that applies not just to the problem at hand, but instead to a larger set of problems.
From a computer science perspective, things that are abstract are also general. Things that are general are also abstract. Substituting the word "general" for the word "abstract" would not be a terrible hurdle.
From a non-computer-science perspective, however, "general" and "abstract" have very different implications. Something that is general is better: it is more useful, it applies more frequently. Something that is abstract, though, is worse: it is lacking detail, it is non-concrete.
This is one difference--the major difference?--between computer science (and of course mathematics) and the real world: the abstract is no less concrete. We can abstract over expressions using functions, and we can even abstract over syntactic things, using hygienic macros. The result of such abstraction is a perfectly well-defined element in our universe of expressions.
In computer science, then, the pejorative sense of the word "abstract" is misleading, and the use of the terms "abstract" and "abstraction" merely provides ammunition for those who wish that we could all still be writing assembly language.
I suggest instead the use of the word "general."
John "purveyor of barbarous neologisms" Clements

Completions in DrScheme (finally)

DrScheme now supports a language- sensitive (but not lexical- scope sensitive) completion feature. Type <menukey>-/ and see what names are available to finish off the word you're typing.

Thanks to Jacob (and do follow that link; we all need a little more love in our lives) and Mike for taking the initiative to actually implement what is probably the most requested feature in DrScheme at the moment.

How many occurrences of car in the PLT source code?

Lets play a guessing game. See who can guess:

• How many occurrences of the identifier 'car' there are in the PLT tree (when using 'read' and just counting the symbols that come out)?
• Where does 'car' rank on the list of the most commonly used identifiers?
• What is the most common identifier, and how many occurrences of it are there?

UPDATE: The two files raw-hattori and raw-kajitani.ss are generated files containing solutions to Paint by Numbers problems and about 30,000 occurrences of x and o. Discounting them, this is the list of the top ten identifiers and the number of occurrences:

((define 25294)
 (quote 24101)
 (lambda 18883)
 (let 14796)
 (send 14349)
 (x 11877)
 (if 11118)
 (... 8474)
 (car 7610)
 (syntax 6537))

The identifier cdr ranks 21st with 5,259 occurrences, let* has 3,066 which, when combined with let comes out at 17,862, still not enough to pass lambda. Speaking of combining, λ has 2,271 occurrences, which is also not enough to move lambda. Finally map comes in 32nd with 3,853 occurrences and foldl beats out foldr (1168th place with 75 occurrences vs 1451st place with 58 occurrences).

Birthday Easter Eggs in DrScheme

DrScheme has five birthday easter eggs in it, one for each of the main contributers to the PLT Scheme infrastructure (Matthias, Matthew, Eli, Shriram, and me). I put four of them in there, and mostly concentrated on making them fun. Matthew added mine and the best part of that one is figuring out on earth it shows up (it is quite tricky to find the code that actually makes that one appear).

I don't want to ruin the fun of searching for the Easter Eggs yourself, but just to get you started, do have a look at plt/collects/framework/private/bday.ss for Matthias, Matthew, Shriram, and Eli's birthdays. Mine is July 2nd.

Happy Hunting!

New Debugger Features

As Eli mentioned, v371 introduces support for debugging several files at a time, as well as new buttons for stepping Over and Out of expressions in the debugger.

Debugging across multiple files is easy. Start by opening the "main" file that you want to debug and all of the files it requires (directly or indirectly) that you want to debug along with it. Then click Debug in the main file's frame. For example, if I wanted to see what the FrTime dataflow engine (in frp-core.ss) does when a particular program (say demo-module.ss) runs, I would open these two files and click Debug in the frame for demo-module.ss.

As each required file loads, DrScheme offers the option of debugging it. If you choose "yes", then the file is included in the debugging session, so you can set breakpoints and step into it. (Note that this will make the code in the file run more slowly, and single-stepping at calls to its functions will bring you into it.) A file can only participate in one debugging session at a time, so if you're already debugging it with some other program, DrScheme will tell you so (instead of asking whether to debug it). For best results, all of the files you debug should be modules. Once a file is included in the debugging session, you can set breakpoints and step into it as if you were debugging it by itself.

As soon as you can debug programs that span several files, it's particularly valuable to be able to do more than set breakpoints and single-step. This is the motivation for the new Over and Out buttons, which are also quite simple. If the execution marker is at the start of an expression that's not in tail position, then you can step over the entire expression, which is equivalent to setting a one-shot breakpoint at the end of the expression and continuing. (If you've set breakpoints inside the expression, or inside any functions it calls, then execution may suspend before reaching the end.) Likewise, if execution is suspended and the current expression is evaluating within a debugging-enabled context, then you can step out to the innermost such context. This would be difficult to simulate by hand, since you'd need to keep track of recent callers.

At any given point, either or both of the Over and Out buttons may be disabled, but over the course of a session they can eliminate a lot of tedium.

The screenshot above shows a session debugging frp-core.ss as used by demo-module.ss. Execution is suspended on a right paren, so stepping Over is disabled, but we see the expression's value at the upper left, we've moused over b to see its value at the upper right, and it's possible to step Out.

PLT Scheme v371

PLT Scheme version 371 is now available from http://download.plt-scheme.org/ This is mostly a bug-fix release. Changes:

• The debugger now works across multiple files and supports "step over" and "step out" operations.
• HtDP teachpacks: the world.ss teachpack now exports two add-line functions: one from image.ss and one for adding lines to scenes.
• ProfessorJ now includes a language level between Intermediate and Advanced, Intermediate + access, that includes all of Intermediate and introduces access modifiers and overloading. The language manuals contain the complete details.

Feedback Welcome.

PLT Modules and Separate Compilation

For my summer job this year, I'm programming in Common Lisp; this is the first time I've used the language for anything more than toy examples. The experience has given me new appreciation for the PLT module system and how it enables separate compilation.

Lisp has a package system, of course, but it's not the same thing. It's primarily a tool to make sure that the symbols in one part of the program don't collide with the symbols in another part (unless you ask them to). Packages aren't about abstraction: while you can specify which symbols are exported from the package and which aren't, that's just a suggestion that's not enforced by the language.

(You'll notice, by the way, that I used the word "symbol" and not "identifier," which is the more common term in the study of programming languages, in the previous paragraph. That's deliberate: the Lisp package system works on symbols, not identifiers, so it also affects quoted, literal symbols. In my experience, this is sometimes helpful, sometimes a real pain, and usually completely unexpected. But that's a topic for another post.)

Also, there's no real relationship between Lisp packages and files. One package can be spread across multiple files, and one file can contain code in several different packages.

All this means that separate compilation in Lisp is a real problem. There is a system, ASDF, that attempts to address this need. (For more details, consult the closest thing to a homepage that I could find for ASDF.) I'm no expert on ASDF, but essentially the programmer specifies the dependencies between source files, in a set of files that exist parallel to the Lisp source. (ASDF does support grouping source files into larger chunks and specifying dependencies between those chunks, but as far as I can tell that's largely a convenience thing.)

The key thing for separate compilation, of course, is the dependencies. With ASDF, the programmer specifies those manually, and then ASDF basically does a topological sort such that if file a depends on file b, then ASDF ensures that a is compiled and loaded before b is compiled, and again before B is loaded. (This should start sounding a little familiar to folks who've worked in the area where PLT's modules and macros intersect.)

So far, so good. Unfortunately, there are a couple of problems with this setup. First, the dependencies between files are specified outside the language. This means that, if you happen to forget one, the results are not well-defined. If ASDF happens to choose an order that's consistent with the dependency you left out, everything will just work, and you won't have any indication that there's a problem. If, however, it doesn't, then you'll get random "undefined function" and "undefined symbol" errors---if you're lucky (at least in SBCL, the implementation of Common Lisp that I use at my job). In PLT, by contrast, inter-module dependencies are part of the language, so the compiler will always give you an undefined-identifier error when it tries to compile a module in which you've forgotten a require form. Big win, in my opinion (although we could argue about whether this should be an error or a warning, and whether the compiler should report lots of errors or just one before giving up completely).

Second, because ASDF lives outside the compiler, it can't be very smart about how macros affect separate compilation. I don't fully understand this, perhaps because the folks who've been mentoring me at my job haven't thought it worth the time to explain it to me fully. But it appears that, if you change a macro that's used in other files, or change a function that's called by a macro at expansion time, you have to do the effect of a make clean in a distressingly large number of cases. This is a real problem when you've got a large source base (~200K LOC, I think) and you're trying to speed up builds, as we are, and it's especially problematic if you're trying to run unrelated parts of the build in parallel.

I've certainly griped about the complexity of the interaction between PLT's modules and macros in the past. But after this summer, I have to say it's awfully nice to have a module system that Just Works for separate compilation. Nicely done, Matthew.

(I've pointed the folks at work at Matthew's ICFP 02 paper, but as that technique requires a lot of support from the compiler, and we don't have the resources to add the necessary support to SBCL ourselves, I don't know that it'll be more than a "wouldn't it be nice if we could do that?")

(Answer to rhetorical question in preceding paragraph: Yes. Yes it would.)