Steering Scheme

Election time is here again. A couple more days and the Scheme community will have a set of new steer-ers.

What should we want from a steering committee?

I have argued at this place before that good language design needs a feedback loop. Language designers write down specs; language implementers translate those specs into compilers and interpreters; programmers use these implementations to produce useful software. The loop comes in when implementers inform designers of flaws, inconsistencies, mistakes, errors, and other internal consistency problems in the specs. This is clearly happening with R6RS, and it is good. Implementers are a biased bunch, however. After all, they work on just one kind of program, and in a highly specialized domain that has been mined for a long time. How can you trust them? [*]

The loop becomes truly useful when people write large software systems (not just compilers, because they really are special cases!) and find that the language fails them in some way. Such failures can come in a number of flavors. For a document such as R6RS, we should hope that programmers can discover problems with porting systems that are apparently due to ambiguities, flaws, mistakes, roaches in the actual document (as opposed to a specific implementation).

So what does this have to do with the steering committee?

The last thing we want from a steering committee is a radical commitment to change (whatever it may be); a prejudice concerning R6RS; a closed mind about the size of "Scheme" (it's too large, it's too small); a willingness to steer without making observations. A steering committee of overbearing curmudgeons is not what we want.

What we do want is a committee that is willing to figure out how the listening is going to happen; how we can possibly finance a systematic way of listening (writing NSF grants, anyone?); how the feedback is best channeled into language design.

Let's hope we get such a steering committee. The Scheme community deserves it.

---

[*] No I am not naive enough to think that language implementers don't design languages, and that implementers don't program systems other than implementations. I am just skeptical that it is easy for them to separate out their various roles in an objective manner, even if many of them are able to think at several different levels about programs.

Don't say "abstract" (instead say "general")

The word "abstract" is common in computer science. An abstract thing is one where some part of the whole is unspecified. For instance, the expression "3*x + 3" is an abstraction of the expression "3*4+3", because the "x" is unspecified. Likewise, a function is an abstraction over some set of values, supplied when the function is called.
The word "general" is not at all common in computer science. In non-computer-science use, the word "general" is used to describe things that may be applied to more than one thing or situation. For instance, a "more general solution" is one that applies not just to the problem at hand, but instead to a larger set of problems.
From a computer science perspective, things that are abstract are also general. Things that are general are also abstract. Substituting the word "general" for the word "abstract" would not be a terrible hurdle.
From a non-computer-science perspective, however, "general" and "abstract" have very different implications. Something that is general is better: it is more useful, it applies more frequently. Something that is abstract, though, is worse: it is lacking detail, it is non-concrete.
This is one difference--the major difference?--between computer science (and of course mathematics) and the real world: the abstract is no less concrete. We can abstract over expressions using functions, and we can even abstract over syntactic things, using hygienic macros. The result of such abstraction is a perfectly well-defined element in our universe of expressions.
In computer science, then, the pejorative sense of the word "abstract" is misleading, and the use of the terms "abstract" and "abstraction" merely provides ammunition for those who wish that we could all still be writing assembly language.
I suggest instead the use of the word "general."
John "purveyor of barbarous neologisms" Clements

Small is Beautiful, Large is Useful, and Scheme is Both

They say, Scheme is small and this is good.

Have you heard of X? No? It is the smallest computational basis. It is a single function that can compute everything a Turing machine can compute; a Church lambda calculus; a Post model; a RAM; a what-have-you model of computation. Indeed, X is so simple that two equations suffice to specify it completely [Barendregt, page 166]. Imagine that: a complete language report in two lines; a compiler that fits in a few K instead of Ms; no more arguments about smallness.

Small alone can't be any good. If you used X alone, your programs would be the size of the universe or something like that. That's what the theory of computability teaches us [Church and Turing]. Adding LAMBDA and a few primitives to get a pure functional language isn't good enough either. That's what the theory of expressiveness shows [Felleisen; Mitchell; Riecke]. And, using an R5RS Scheme to build large systems with many people at a dozen sites isn't doable either. That's what the PLT experience determined.

When we set out to construct DrScheme using MzScheme, we also conducted an experiment:

Could we really build a graphical system that manages (shared) resources and that provides excellent error feedback with just plain Scheme?

Could we just add enough libraries to do all this? Or would we have to change the kernel of the language? As much as we tried to keep MzScheme small, it became clear quickly that we needed exceptions, structures, module-like features, a mechanism for concurrency, a way to manage resources such as windows, tcp connections, and so on. The list isn't infinite but it is much longer than I expected. Our "Revenge of the Son of the LISP machine" paper is a good summary for the state of the art around 1999 [Flatt and Son].

As language designers we stepped back time and again to look at our monster. What could we remove? What would we have to add in response? For some five years, we had first-class modules (units) and first-class classes in the core of the language. We had almost banned macros. They were so ugly I stopped teaching about them because I did want to use our own dog food in my courses but I couldn't stomach the macro system. It was such a step back from Eugene's extend-syntax. But then Matthew figured out the next big step in macro and module technology [Flatt, You Want It When?]. And with that out went units and classes from the core and many other things. So we learned lessons, and we need to keep building systems to learn more.

I have no question that the idea of Scheme is beautiful. At the same time, I have also learned that if I wish to use this beautiful idea in practice, I need to add the ingredients that it takes to build large systems. R6RS reflects this insight, and I am happy about it.

R6RS is "perfect"

When I read the "side by side" and "head to head" descriptions of the alternatives facing the Scheme community (see Comp.Lang.Scheme and the R6RS mailing list), I am wondering which one is which and which one is better.

• Is it really good that Scheme (the spec) doesn't support a module system?
• Is it really good that almost all major implementations support their own version of a module system?
• Is it really good that programmers can't even leave the module structure intact when porting code?

Imagine your own similar questions and add them here. We have lived in a side-by-side universe for a long time, and there are quite a few programmers who have suffered from this not-really-the-same-language problem. Besides the module system, there are other not-quite-the-same-but-related features that implementations have and programmers wish to use.

The R6RS process has pushed several major implementors/implementations to agree on a design for module systems and other constructs. Their report declares that they are ready to put a large amount of work in to get from r5rs to r6rs. I believe that this step would help the community in several arenas, listed in increasing order of relevance:

1. the academic publishing business
2. the fund raising business
3. adapting each others innovations
4. supporting programmers who learn on one and switch to another implementation
5. supporting commercial programmers who need reassurance that there is more than one implementation and implementor [ever attended Commercial Uses of Functional Programming?]

Is the document perfect? Is every construct exactly the 'right thing'? Of course not! Guy and Gerry revised their first Scheme report because they didn't get it 'right'. R3RS and R4RS and R5RS revised flaws in R(n-1)RS because the authors/editors didn't get it 'right'. It is extremely difficult, and usually impossible, to get the design of a complex artifacts (such as a programming language) 'right' the first time. In these cases, it's all about the feedback loop and revising your design based on observations. (Remember the 'science' part in the name of our discipline?) Indeed, 'right' doesn't exist; what exists is 'most pragmatic and internally beautiful,' and nothing else.

Our choice is quite simple: move forward as a community with some amount of convergence (r6rs) or split into dozens of mutually incompatible sub-communities (status quo, including SRFIs).

Also posted as "R6RS is perfect" at the R6RS discussion list.

Macros Matter

Thank you Jens for setting up this Blog.

PLT Scheme is a 12-year old project now and it is definitely time to open it up to the world. The language and the project has contributed numerous ideas and products to the world. This covers programming languages (units, mixins, an implementation of cml-style concurrency, etc); programming tools (drscheme, check-syntax, transparent repls, module browsers, etc), programming pedagogy (htdp, htdc); program engineering (we resurrected the "expression" problem, web programming and continuations); and some more.

Time and again, people have asked me what I consider the one 'feature' that distinguishes us from the rest of the hordes of programming languages. I always respond with a single word:

                                            macros.

We have pushed macros hard, and we have accomplished a lot with them. I conjecture that without macros, we would never have achieved the level of productivity that this group displays.

Of course, everyone else in academia works on types. ML's module type system of the third kind and Haskell's system-complete type system are serious challenges to anyone. It is probably true that you shouldn't consider yourself a programmer if you can't read and write some of those type-laden programs, and I seriously believe that they are the next generation of influential languages.

For the generation-after-the-next then, I see "macros" as one of the big topics (next to concurrency). A real programmer will have to know how Lisp and Scheme-style macros can reduce labor by orders of magnitude, how macros provide the tools for creating the "ultimate abstraction" in the form of domain-specific and embedded languages (Hudak's words). And there is no better place to start with than PLT Scheme's macro system.

So I would like to dedicate this blog to all things macros and everything else that matters in (and to) PLT Scheme.