This is a direct continuation of Programming Language Study–Lisp (2).
After my last post about Lisp, several people on Twitter wrote to me with a lot of helpful feedback about what I was doing wrong. I have not read and studied everything they wrote yet, so that is the topic of today’s study. I will share the ideas with you guys and I’ll comment on them and illustrate the ideas as I study.
Some of the stuff I wrote earlier was not technically correct—obviously. I got errors that I could not fix. If I already understood Lisp I’d be writing a different kind of tutorial. (And also I wouldn’t be writing that tutorial. I hate writing tutorials about some topic I already know. It’s boring.)
A man on Twitter named Rainer Joswig (@RainerJoswig) was very helpful in understanding what was going wrong with my attempt to define a custom-defun function. The conversation is all publicly visible on Twitter, so if you want to do so, you can read our conversation directly. It is somewhere underneath this tweet: the tweet where Rainer Joswig offers to answer questions. Now, let’s go through what he taught me.
First of all, I thought that the built-in operator DEFUN was a function, but it is not. It’s something called a macro. If you are new to Lisp like me but you have heard the word “macro” in some other context, I advise you to forget that and come at this with a clear mind. It’s probably not like an Excel macro or a C macro.
I used the word “operator” above because I needed a word more generic than “function” since in this context “function” and “macro” both seem to have specific meanings that I need to keep track of. I don’t believe “operator” has any very explicit definition in this context. It means sort-of “a thing that operates on other things”. Anyway, that is how I intend to use the word: vaguely.
That said, we do need to understand what a Lisp macro is, and what a Lisp function is, and the differences between them. I’m going to read on in the tweets and see if I can understand.
Joswig says that if I want to define my own custom-defun then either I can write my own macro, or I could write a function, but as a function it will have to work differently from the way the built-in DEFUN macro works.
He has transformed my attempt into a function, I think. He wrote the following: (defun custom-defun (name args definition) (register name) (setf (symbol-function name) (eval `(lambda ,args ,@definition)))). I am honored that he used my example and turned it into something that works. At this moment, of course, I do not understand what he wrote, so I will try to break it down.
Last time while I was trying to figure out how to make this thing work, I came across the single-quote ' notation in use. My understanding of that notation is that it indicates that a list should be passed as a list instead of being evaluated and passing the value.
In my previous attempt, I used PROGN to execute two different operations in sequence. It looks like that was unnecessary, but at the time it seemed like I needed to do that. Based on the example Joswig wrote, it looks like you can execute operations in sequence by just arranging them as items in the top-level list as arguments going into DEFUN. Let’s make a note to try that ourselves later using a different example. It will have to be something where the first operation causes a side effect or else there will be no way to test that it was executed.
I don’t know the setf symbol, so I will Google that. I found what looks like a useful answer here: http://www.n-a-n-o.com/lisp/cmucl-tutorials/LISP-tutorial-16.html. I don’t pretend to fully understand. At this point I don’t even know if setf is a function or a macro. But based on that tutorial I linked, it looks like (setf x y) evaluates the expression x to find a location in memory and then evaluates y for a value and stores that value in the location of x.
Ahhh. I think I’m starting to understand how Mr. Joswig’s function works. I’m going to guess that the expression (symbol-function name) would find the location in memory of a function having the name name (that is: not literally “name”, but the value contained in the variable name). Thus, that is the first argument going into setf. The second argument is an expression of the function going into that memory slot.
I don’t know the specifics yet, but I can guess what eval means, and I know that in programming lambda usually refers to an anonymous function. So my guess is that this expression defines an anonymous function that will be inserted into that memory slot.
I still do not know what is going on with the commas or the at @ symbol, so let me Google around for that.
I found this page, which looks useful: https://stackoverflow.com/questions/21463925/meaning-of-at-sign-in-lisp. It explains what the ,@ symbol does. Note that here it’s not the at-sign alone that has a special meaning. It’s a comma followed by the at-sign. That is something explained in the StackOverflow QA just linked. It looks like this symbol takes a list as input (the symbol immediately to the right of it) and it has the effect of splicing this list into whatever list the ,@ occurs in lexically.
However, I ran an experiment on my Lisp REPL to verify this. I executed (1 2 ,@(3 4)), and it complained “Comma not inside backquote”. So let’s keep reading to better understand what backquote it’s talking about.
I do note that Mr. Joswig’s function definition above contains a backtick—otherwise called a backquote I guess. So let’s see what a backtick means in Lisp. I found the answer in this page: https://stackoverflow.com/questions/30150186/what-does-backtick-mean-in-lisp. I’ll rephrase the answer slightly.
If you put a single quote ' in front of an expression, then the evaluation of that is the literal expression without any evaluation. So '(1 x "foo") evaluates to (1 X "foo"). That is literally what it prints on the screen.
I assume that console output means that the value is the list (1 X "foo"), not the string that looks like it. The thing to note here is that you get X, not the value of X, even if it happens to have a value in this scope. Basically this is like a list literal, but in our case we want a kind of “list interpolation”. That is—we want to specify the format of a list and pass in placeholders that will be evaluated to produce the list, sort of like how the C printf function allows you to insert the values of variables into a string. The single quote ' does not offer any kind of interpolation, but the backtick does. If you write `(1 X "foo") then you can put a comma in front a symbol to indicate interpolation. Thus `(1 ,X "foo") would evaluate to a list where ,X was replaced with the value in scope of X.
Now, let’s see if we can make sense of (eval `(lambda ,args ,@definition)). I think the eval function was needed in order to interpolate the list inside the backtick.
We want the value of that expression after args has been replaced by the value of args in scope, and after ,@definition has been evaluated. I’m pretty sure from context that lambda just indicates an anonymous function. I’m still a little confused as to why we want to splice in the value of definition instead of just inserting it right there as a list. But I have a guess.
My guess about definition is that if you define an argument list in the form (a b c) and then you pass four or more arguments into the function then it includes arguments 3+ in a list in place of argument c. That is a common language construct. It’s how variable argument functions are written. I think in order to test that I would need to know how to define a function that loops over elements in a list, and I have not learned how to loop over a list yet, so…let’s make a note to do that some other time.
I think we have understood that deep enough for now. Let’s try actually evaluating it on the Lisp REPL and see if we can make it work. I don’t want to define a “register” function right now, so I’m going to just put in a placeholder that does nothing. This is the code exactly as I will execute it.
(defun custom-defun (name args definition) (+ 2 3) (setf (symbol-function name) (eval `(lambda ,args ,@definition))))
This works, and it will allow us to define a function using a function, but it kinda sucks and I’ll show you why.
Wait…hold on. I accidentally clicked on some text near the bottom of the Emacs window and it switched me to some environment where you can’t do anything. I think I’m looking at a log output now. Ah, I bet that is a different “buffer”. OK, I was able to switch back to the slime-repl buffer to get the command line back.
I executed the code just above to define custom-defun. Now I need to use it to define something so I can test it. This is where it sucks. I’m mimicking Mr. Joswig’s example as I write (custom-defun 'add-them '(x y) '((+ x y))). What sucks is all the quotes needed, and having to double-wrap the definition as a list inside a list. After all of this, it works. The expression (add-them 2 3) evaluates to 5 as expected, but it is confusing. It does not parallel the functionality of the real DEFUN macro. Really what I want is something that mimics the way DEFUN works and is written the same way but also has extra side effects that are not built into DEFUN. Well—I don’t actually need that, but that is the example I set out to study. So next time we will learn macros.
I was going to include macros in this post, but it already feels like enough. So…next time. See ya.
adamcross.blog 