;;; LISP and Scheme pioneered the concept of _closures_: packaging the memory state of a ;;; function at the point of its dynamic creation, as a first-class programming ;;; language construct. ;;; So not only are LISP functions first-class values that can be passed around as ;;; arguments to other functions (most commonly, MAP, REDUCE, or other kinds of folds), ;;; but so is the relevant state of the environment in which they are being created. ;; This function returns a copy of an unnamed function _with its very own_ counter: ;; (defun make-counter () (let ((cnt 0)) (lambda () (setq cnt (1+ cnt)) cnt))) make-counter (setq a (make-counter)) (closure ((cnt . 0)) nil (setq cnt (1+ cnt)) cnt) (funcall a) 1 (funcall a) 2 ;;; We can create a different function b, whose state is independent of a: (setq b (make-counter)) (closure ((cnt . 0)) nil (setq cnt (1+ cnt)) cnt) (funcall b) 1 (funcall a) 3 a (closure ((cnt . 3)) nil (setq cnt (1+ cnt)) cnt) ;;; A brief interlude: ;;; FUNCALL is a clumsy way to call functions, so we'll rescue this binding into an easily ;;; callable form, by binding it to a's symbol-function slot (it's now in symbol-value slot) (setf (symbol-function 'a) a) (closure ((cnt . 3)) nil (setq cnt (1+ cnt)) cnt) (a) ; so now we can call it 4 a (closure ((cnt . 4)) nil (setq cnt (1+ cnt)) cnt) (setq a 'foobar) ; we can change the symbol-value to whatever we want, and (a) will continue foobar ; to be callable (a) ; symbol-function 5 a foobar ; symbol value ;;;;; end interlude ;; Note that a and b have separate states, although they both share the same code. ;; You can think of them as objects, specifically different _instances of an object type_. (b) (funcall b) 2 b (closure ((cnt . 2)) nil (setq cnt (1+ cnt)) cnt) (funcall b) 3 b (closure ((cnt . 3)) nil (setq cnt (1+ cnt)) cnt) ;;;; Let's see how these closures are made! ;; Note that eLisp uses a very transparent way of implementing closures: it ;; explicitly creates new cons cells to point to copies of objects closed over ;; to form the lambda's private state. Other LISPs may use different structs to ;; represent these private environments and make them less visible, but the idea ;; is illustrated by eLisp very cleanly. ;; If you haven't read the blogpost about eLisp's byte-compilation, read it now: ;; https://nullprogram.com/blog/2014/01/04/ (byte-compile 'make-counter) #[0 "\300C\301\302\"\207" [0 make-closure #[0 "\300\211\242T\240\210\300\242\207" [V0] 2]] 4] ;; Note the make-closure function. It will be called to make the actual closure. ;; It's not magic: https://mbork.pl/2016-05-17_Emacs_Lisp_closures_demystified shows ;; exactly how it works, in a way that is discouraged by the Emacs manual :) ;;; Just in cаse you wonder about the second string of byte-compiler code, it's the LAMBDA ;;; with its LET: (byte-compile (let ((cnt 0)) (lambda () (setq cnt (1+ cnt)) cnt))) #[0 "\300\211\242T\240\210\300\242\207" [(0)] 2] | +------------------------------------------------------------------+ | | (disassemble-internal 'make-counter 2 nil) | args: nil | 0 constant 0 ; [ 0 ] | 1 list1 ; [ (0) ] a new list, will be the closure's private storage | 2 constant make-closure ; [ make-closure (0) ] | 3 constant ;; <-----------------------------------------+ args: nil 0 constant V0 ; [ V0 ] <-- V0 is a "closure var reference", ; will point to the list created by list1 ; after the make-closure call below is done 1 dup ; [ V0 V0 ] 2 car-safe ; [ 0 V0 ] ; 0 on the first run, 1 on the next, etc. 3 add1 ; [ 1 V0 ] ; 1 on the first run, 2 on the next, etc. 4 setcar ; [ 1 ] ; sets the CAR of the cons cell referenced by V0, created by list 1 as above 5 discard ; [ ] ; discards the result of SETCAR 6 constant V0 ; [ V0 ] ; back to list1-created list/cons cell 7 car-safe ; Gets the value from list1-created cons cell. Non-optimal, since ; but just discarded the exact same value, but 8 return ; we've seen such redundancy in assembly, too. ;; the stack is now [ "\300\211\..." make-closure (0) ] 4 stack-ref 2 ; [ (0) "\300\211\..." make-closure (0) ] ; the newly created cons cell is now in position 5 call 2 ; now we have a closure over that cons cell, packaged with the bytecode 6 return ; and we return it nil (setq c (make-counter)) (closure ((cnt . 0)) nil (setq cnt (1+ cnt)) cnt) ;; As expected, the closure/LAMBDA functions created by MAKE-COUNTER ;; have identical bytecode but different cons cells and values that ;; they closed over when created: (disassemble-internal c 2 nil) args: nil 0 constant (0) ; <--- note this list i.e. cons cell, a permanent memory address associated with this LAMBDA instance 1 dup 2 car-safe 3 add1 4 setcar 5 discard 6 constant (0) 7 car-safe 8 return nil (funcall c) 1 c (closure ((cnt . 1)) nil (setq cnt (1+ cnt)) cnt) ;; The disassembler will helpfully show the current content of the closure's cons cell. ;; Remember that the byte code at the start of the lambda remains the same, \300, i.e., ;; "push the reference to the first element of the const-vector associated with the code ;; to the top of the stack". It's just the content of that element that changes. (disassemble-internal c 2 nil) args: nil 0 constant (1) ; ... and now the CAR of this associated "closure" cons cell has changed 1 dup 2 car-safe 3 add1 4 setcar 5 discard 6 constant (1) 7 car-safe 8 return nil (funcall c) 2 (disassemble-internal c 2 nil) args: nil 0 constant (2) ; ... and again, same bytecode, different value in the cons cell 1 dup 2 car-safe 3 add1 4 setcar 5 discard 6 constant (2) 7 car-safe 8 return nil ;;; And thus do the closures work! Now re-read ;; https://mbork.pl/2016-05-17_Emacs_Lisp_closures_demystified :)