slme-profile-package had shown that practically all of the running time was spent in one function that did a lot of list building, and that this function was consing a LOT of memory. I tried to change the appends to nconcs, and after a bit of fiddling the profiling reported that the function was only consing about half as much memory as before. The program also runs a nice bit faster.
The problem I had to fix was that I had some list literals in code, that nconc–true to its nature– was destructively modifying. When recreating a minimal case here SBCL is nice enough to warn me about it, but I don’t recall seeing this when originally solving the problem. There was a lot of other code inbetween that may have been breaking its inferencing though. Another thing to notice is sbcl decided that the length of a constant list was constant=3, except in this case the list isn’t as constant as it appears to be. :-)
CL-USER> (defun foo (a)
(let ((b '(1 2 3)))
(values (length b)
(nconc b a))))
; in: LAMBDA NIL
; (NCONC B A)
;
; caught WARNING:
; Destructive function NCONC called on constant data.
; See also:
; The ANSI Standard, Special Operator QUOTE
; The ANSI Standard, Section 3.2.2.3
;
; compilation unit finished
; caught 1 WARNING condition
STYLE-WARNING: redefining FOO in DEFUN
FOO
CL-USER> (foo '(bam baz))
3
(1 2 3 BAM BAZ)
CL-USER> (foo '(bam baz))
3
(1 2 3 BAM BAZ BAM BAZ)
CL-USER> (foo '(bam baz bad))
3
(1 2 3 BAM BAZ BAM BAZ BAM BAZ BAD)
CL-USER> (defun foo (a)
(let ((b (list 1 2 3)))
(values (length b)
(nconc b a))))
STYLE-WARNING: redefining FOO in DEFUN
FOO
CL-USER> (foo '(bam baz bad))
3
(1 2 3 BAM BAZ BAD)
CL-USER> (foo '(bam baz bad))
3
(1 2 3 BAM BAZ BAD)