How do you securely parse untrusted input in Common Lisp?

Question

How do you securely parse untrusted input in Common Lisp? Given that there is no parse-float etc, and that read-from-string will execute reader macros like #. (read time eval).

e.g. (read-from-string "#.(+ 1 2)") => 3

Khabarovsk

Note that you can disable #. with *read-eval*. lispworks.com/documentation/HyperSpec/Body/02_dhf.htm. That said, it's hard to be completely safe, since there are other kinds of macros that can be invoked by the reader. Rather recently, someone wrote up some good notes about safe input processing in Common Lisp (looking for it now). – Sheen

Also look into with-standard-io-syntax, as mentioned in #32492697 . – Sheen

I understand that this is an example, but if you only want to parse floats, see crategus.com/books/parse-number – Douceur

"Given that there is no parse-float etc, " is also a bit broad. There are some parsing functions, just not all of them. E.g., there's a standar parse-integer. Implementations might provide others, too. E.g., Lispworks does have a parse-float. – Sheen

Answer 1

Generally, just that the standard code reader is so readily available and can read many kinds of input does not mean that you should use it to read anything but code.

There are many libraries for parsing a lot of things, e. g. parse-number for the Lisp number formats, fare-csv for CSV files (among many other CSV libraries), json-streams for JSON (again, many others). For most formats, you can just do a system-apropos lookup with Quicklisp.

Creosol

Answer 2

I can't find the other question or comment that described some of the safe input handling procedures for Common Lisp (if someone else finds them, please post a comment!), but there are at least two important things that you might do:

• Use with-standard-io-syntax to make sure that you're reading with the standard readtable, etc. Note that this will bind *read-eval* to true, so be sure to also:
• Bind *read-eval* to false (within with-standard-io-syntax). This disables the sharpsign-dot (#.) macro mentioned in the question.

(let ((*readtable* (copy-readtable)))
  (set-macro-character #\n (constantly 'injected))
  (read-from-string "(#.(+ 2 5) n)"))
;;=> (7 INJECTED)

(let ((*readtable* (copy-readtable)))
  (set-macro-character #\n (constantly 'injected))
  (with-standard-io-syntax
    (let ((*read-eval* nil))
      (read-from-string "(#.(+ 2 5) n)"))))
;; Evaluation aborted on #<SB-INT:SIMPLE-READER-ERROR
;; "can't read #. while *READ-EVAL* is NIL" {1004DA3603}>.

(let ((*readtable* (copy-readtable)))
  (set-macro-character #\n (constantly 'injected))
  (list (read-from-string "(n)")
        (with-standard-io-syntax
          (let ((*read-eval* nil))
            (read-from-string "(n)")))))
;; ((INJECTED) (N))

Sheen

Make sure that you rebind *read-eval* to NIL inside with-standard-io-syntax, because the latter will bind it to its default value T!!! – Cobbs

@RainerJoswig Reordered bullet items, with some explanation. – Sheen