(Chez) Scheme benchmarks?

Question

Now that Chez Scheme is open-source, I wonder how it compares to Racket and other Schemes or languages in terms of performance, so that one could make informed choices about using them in one's projects.

Unfortunately, I couldn't find any relevant benchmarks.

I found the following:

---

https://ecraven.github.io/r7rs-benchmarks/benchmark.html

Problem: no Racket, or other languages (Update 10/13/18: Chez is now included in some of the benchmarks)

---

http://www.larcenists.org/benchmarksGenuineR6Linux.html

Problem: no Chez Scheme, or other languages

---

https://benchmarksgame-team.pages.debian.net/benchmarksgame/

Problem: only Racket, questionable comparisions (For example, Python is not allowed to use Numpy where it would clearly help, while Racket is making FFI calls to GMP)

---

So, none of the benchmarks I found allow you to compare Racket to Chez, for example, or Chez to SBCL, or Java. Are there Chez benchmarks that give you a sense of how fast it is?

Chez Scheme is often said to be the fastest Scheme/Lisp around. We should know if it's faster than, say, Java for your typical business logic application.

Chiliad

So what is it ou want to test R6RS, R7RS? Have you considered just downloading the top level test, compile it in the two and run them? – Symbolism

>>while Racket is making FFI calls to GMP<< Is Python allowed to make FFI calls to GMP? – Adversity

@Adversity Why are you asking me? Have you specified the rules of the benchmark, before renaming this thing into a "Game"? Why would you forbid using popular idiomatic libraries like Numpy and, at the same time, let people call any external C/Asm libraries? – Chiliad

@MaxB Just like Racket, Python is allowed to make FFI calls to GMP; and there's a Python program that does. – Adversity

There is a bunch of Common Lisp benchmarks. Some of them should have been ported to Scheme. This project assembles common-lisp.net/project/cl-bench some of the benchmarks. The early Lisp benchmarks were the 'Gabriel Benchmarks', here included. Those are widely translated. – Franzen

@MaxB >>and, at the same time, let people call any external C/Asm libraries?<< There were language implementations that implemented arbitrary precision by bundling GMP, so it seemed unfair not to allow other language implementations to use GMP if they could. Level playing field. – Adversity

It should be noted that the first link of this question now includes Racket benchmarks. – Vasilikivasilis

@Vasilikivasilis good to know. I guess one could use that to compare Racket to Chez, and the "not shootout" project to compare Racket to Java, C and SBCL. – Chiliad

all, please take the data shown on benchmarksgame site with a lot of salt (not just a pinch). I have it on very good authority that many of those results are not true: try out the codes by yourselves to make sure. – Slipslop

Answer 1

Kent Dybvig has written articles on the implementation Chez Scheme. They'll often have comparisons with other implementations:

https://www.cs.indiana.edu/~dyb/

Hoist

Answer 2

It's anecdotal, but Matthew Flatt, the lead developer of Racket, thinks Chez is pretty good. You can read more about it here. He cites a regular expression matcher in which Chez is twice as fast as Racket and comparable to C.

Willywilly

I'd seen those. "Everyone" thinks Chez is pretty good, but is it in the same league as Java, or at least SBCL? – Chiliad

I don't think you'll find a controlled study, only anecdotes. But Matthew Flatt is hardly "everyone." If he thinks Chez is sufficiently better than Racket to abandon the native compiler in Racket in favor of Chez, I'm inclined to believe him. My experience is that Chez is as fast as anything else, and also rock-solid, in terms of not having bugs, which is at least as important as raw speed. – Willywilly