Why is there no generic operators for Common Lisp?

Question

In CL, we have many operators to check for equality that depend on the data type: =, string-equal, char=, then equal, eql and whatnot, so on for other data types, and the same for comparison operators (edit don't forget to answer about these please :) do we have generic <, > etc ? can we make them work for another object ?)

However the language has mechanisms to make them generic, for example generics (defgeneric, defmethod) as described in Practical Common Lisp. I imagine very well the same == operator that will work on integers, strings and characters, at least !

There have been work in that direction: https://common-lisp.net/project/cdr/document/8/cleqcmp.html

I see this as a major frustration, and even a wall, for beginners (of which I am), specially we who come from other languages like python where we use one equality operator (==) for every equality check (with the help of objects to make it so on custom types).

I read a blog post (not a monad tutorial, great serie) today pointing this. The guy moved to Clojure, for other reasons too of course, where there is one (or two?) operators.

So why is it so ? Is there any good reasons ? I can't even find a third party library, not even on CL21. edit: cl21 has this sort of generic operators, of course.

On other SO questions I read about performance. First, this won't apply to the little code I'll write so I don't care, and if you think so do you have figures to make your point ?

edit: despite the tone of the answers, it looks like there is not ;) We discuss in comments.

Grossularite

Generic CLOS functions were added several years after CL was originally designed (82-84). The variant with CLOS was widely published with CLtL2 (1990) and then ANSI CL. The language was only slightly updated. It was not the intention to make Common Lisp fully object oriented. Also performance of CLOS for relatively low-level functions is kind of problematic. Dylan, which is something like Scheme + CLOS - s-expression syntax, did this: it defines more of the language in terms of generic functions. – Mcguigan

Answer 1

A newer library adds generic interfaces to standard Common Lisp functions: https://github.com/alex-gutev/generic-cl/

GENERIC-CL provides a generic function wrapper over various functions in the Common Lisp standard, such as equality predicates and sequence operations. The goal of the wrapper is to provide a standard interface to common operations, such as testing for the equality of two objects, which is extensible to user-defined types.

It does this for equality, comparison, arithmetic, objects, iterators, sequences, hash-tables, math functions,…

So one can define his own + operator for example.

Grossularite

Answer 2

Kent Pitman has written an interesting article that tackles this subject: The Best of intentions, EQUAL rights — and wrongs — in Lisp.

And also note that EQUAL does work on integers, strings and characters. EQUALP also works for lists, vectors and hash tables an other Common Lisp types but objects… For some definition of work. The note at the end of the EQUALP page has a nice answer to your question:

Object equality is not a concept for which there is a uniquely determined correct algorithm. The appropriateness of an equality predicate can be judged only in the context of the needs of some particular program. Although these functions take any type of argument and their names sound very generic, equal and equalp are not appropriate for every application.

Specifically note that there is a trick in my last “works” definition.

Property

oh right, good reminder. But does it work for lists, vectors, hash-tables and other common types, and can we define its equality for another type, say an fset ? – Grossularite

@Grossularite I edited the answer for equality testing of lists, vectors and hash tables. I understand the hunger for a more extensible == though. – Property

So if I understand the article: there are many operators because they do different things and because the program doesn't read the programmer's head (also for historical reasons). He proposes new verbs (he does not subclass existing ones, they are not CLOS based and thus not extensible like in python) by using defgeneric. There, the type is by default guessed with the arguments, but it can also be given as an optional parameter. It's only a POC and a package still has to be written then ! – Grossularite

Mostly yes. My reading is that the proposed new verb is a defun and that it is implemented in terms of defgeneric/defmethod. A eye opening part is “When [some programmers are] urged to write their own equality predicate to suit their particular needs, they sometimes react as if we are putting them off, rather than realizing that any function they could write is just as valid as any one the language provides. Were the language changed to accommodate such bug reports, different users would probably complain” – Property

Answer 3

Yes we have! eq works with all values and it works all the time. It does not depend on the data type at all. It is exactly what you are looking for. It's like the is operator in python. It must be exactly what you were looking for? All the other ones agree with eq when it's t, however they tend to be t for totally different values that have various levels of similarities.

(defparameter *a* "this is a string")
(defparameter *b* *a*)
(defparameter *c* "this is a string")
(defparameter *d* "THIS IS A STRING")

All of these are equalp since they contain the same meaning. equalp is perhaps the sloppiest of equal functions. I don't think 2 and 2.0 are the same, but equalp does. In my mind 2 is 2 while 2.0 is somewhere between 1.95 and 2.04. you see they are not the same.

equal understands me. (equal *c* *d*) is definitely nil and that is good. However it returns t for (equal *a* *c*) as well. Both are arrays of characters and each character are the same value, however the two strings are not the same object. they just happen to look the same.

Notice I'm using string here for every single one of them. We have 4 equal functions that tells you if two values have something in common, but only eq tells you if they are the same.

None of these are type specific. They work on all types, however they are not generics since they were around long before that was added in the language. You could perhaps make 3-4 generic equal functions but would they really be any better than the ones we already have?

Narceine

Good answer! The point is that different equality operators have different semantics, and the system cannot choose automatically which is the right one for a certain situation (since it does not read the mind of the programmer to know his needs...). – Doited

I also wonder if this is not too a newbie question, if in practice we don't care about using a specific operator because we know the type used in our code, and if we should know it anyway or if it is a pain for exploratory programming, and/or if knowing the type and using the right thing helps in our architecture and the compiler to find bugs. Any experience to share ? – Grossularite

EQ does not work well for numbers or characters because “Common Lisp makes no guarantee that eq is true even when both its arguments are ``the same thing'' if that thing is a character or number” (lispworks.com/documentation/lw51/CLHS/Body/f_eq.htm). You will be better served by EQL. – Property

@Property The question is about generic and eql is less generic than equal and equal can be used all place eql will be t for the types you mention. If you read my answer you know that (eq 1 1) might not do what you want since they are really different objects that look the same (in some implementations, not all) – Narceine

so it looks like there is an equality operator that does a lot. What about comparators, like > ? Do we have a generic one that works on numbers, strings, characters, sequences, other data types like sets, and can we make them work for an object of ours ? – Grossularite

@Grossularite There is nothing stopping you from making it a generic and making a lessp that works for two objects of the same type, but I imagine you need more of those too since sometimes you want (lessp "t" "T") to be t and sometimes they are equal and thus nil if you disregard case. And how about (lessp 2.0 2)? Since 2.0 is somewhere ebtween 0.96 and 2.04 surely it's t half of the time? – Narceine

To me the most troubling part about eq with numbers and characters is that the result of (let ((x 0)) (eq x x)) is unpredictable. – Property

@Property (let ((x 0)) (eq x x)) is always guaranteed to be t. (eq 0 0) is not. You see the difference is that each time the reader reads a literal it can create an entirely new object and thus not guarantee pointer equalness, but two bindings to the same read literal or values created by the same lexical place at one time in one thread is unique and eq with itself always. – Narceine

This is not my reading of the standard: (let ((x 5)) (eq x x)) => true OR=> false and “An implementation is permitted to make "copies" of characters and numbers at any time. The effect is that Common Lisp makes no guarantee that eq is true even when both its arguments are "the same thing" if that thing is a character or number.” (lispworks.com/documentation/lw51/CLHS/Body/f_eq.htm) – Property

@Property Thanks!! I' totally baffled that such enormous detail is so subtly hidden in the hyperspec. CLtL mentiones it using half a page explaining this and why. I states it gives "tremendous performance gains in many common situations". I feel this merits a question on its own since there are probably many implementation maintainers that can confirm this and perhaps have examples. – Narceine

Answer 4

Fortunately CL21 introduces (more) generic operators, particularly for sequences it defines length, append, setf, first, rest, subseq, replace, take, drop, fill, take-while, drop-while, last, butlast, find-if, search, remove-if, delete-if, reverse, reduce, sort, split, join, remove-duplicates, every, some, map, sum (and some more). Unfortunately the doc isn't great, it's best to look at the sources. Those should work at least for strings, lists, vectors and define methods of the new abstract-sequence.

see also

• https://github.com/cl21/cl21/wiki
• https://lispcookbook.github.io/cl-cookbook/cl21.html

Grossularite