Simple Test Coverage: A Macro with Line Numbers and Lifting

Racket’s macro system makes it easy to roll your own low-tech line coverage tool. In this post, I’ll show how, in 15 lines of code, you can implement a simple test-coverage tool. Using this code is simple: put (line-of-interest) on each line that should be covered.

To start the implementation, we put the code in a module and define two sets:

#lang racket

(define candidate-lines (set))

(define touched-lines (set))

The first set holds the line numbers where (line-of-interest) is written in the source and the second holds the set of line numbers where (line-of-interest) has been executed.

Each use of (line-of-interest) is going to expand into a call to visited with the line number for the source location of that use of (line-of-interest).

(define (visited line)

(unless (set-member? touched-lines line)

(set! touched-lines (set-add touched-lines line))

(displayln

(sort (set->list

(set-subtract candidate-lines touched-lines))

<))))

This function simply checks to see if this line has been executed before and, if not, removes that line number from touched-lines and prints out the current status.

The interesting part of this code is in the definition of line-of-interest itself:

(define-syntax (line-of-interest stx)

(with-syntax ([line (syntax-line stx)])

(syntax-local-lift-expression

#'(set! candidate-lines (set-add candidate-lines line)))

#'(visited line)))

The macro first extracts the line number from stx, which gives the source location for the use of (line-of-interest). This number is then bound to line for use in building later syntax objects. Then the macro calls syntax-local-lift-expression with a syntax object that updates candidate-lines. Expressions passed to syntax-local-lift-expression are lifted to the top-level of the enclosing module making sure that, in this case, each line number is added exactly once without having to execute the code where (line-of-interest) appears. The macro then discards the result of syntax-local-lift-expression and returns a call to the visited function. That’s all there is to it!

I originally used this macro to test some changes to DrRacket. I was working on a set of complex GUI interactions and kept losing track of which ones had been tested and which ones hadn’t. Here’s a simpler program in the same spirit so you can try it out.

#lang racket/gui

(define candidate-lines (set))

(define touched-lines (set))

(define (visited line)

(unless (set-member? touched-lines line)

(set! touched-lines (set-add touched-lines line))

(displayln

(sort (set->list

(set-subtract candidate-lines touched-lines))

<))))

(define-syntax (line-of-interest stx)

(with-syntax ([line (syntax-line stx)])

(syntax-local-lift-expression

#'(set! candidate-lines (set-add candidate-lines line)))

#'(visited line)))

(define f (new frame% [label ""]))

(define b1 (new button%

[label "1"]

[parent f]

[callback

(λ (a b)

(case (random 3)

[(0)

(line-of-interest)

(send b1 set-label "one")]

[(1)

(line-of-interest)

(send b1 set-label "uno")]

[(2)

(line-of-interest)

(send b1 set-label "一")]))]))

(define b2 (new button%

[label "2"]

[parent f]

[callback

(λ (a b)

(case (random 3)

[(0)

(line-of-interest)

(send b2 set-label "two")]

[(1)

(line-of-interest)

(send b2 set-label "dos")]

[(2)

(line-of-interest)

(send b2 set-label "二")]))]))

(send f show #t)