Which scripting languages support multi-core programming?

Question

I have written a little python application and here you can see how Task Manager looks during a typical run.
_{(source: weinzierl.name)}

While the application is perfectly multithreaded, unsurprisingly it uses only one CPU core. Regardless of the fact that most modern scripting languages support multithreading, scripts can run on one CPU core only.

Ruby, Python, Lua, PHP all can only run on a single core. Even Erlang, which is said to be especially good for concurrent programming, is affected.

Is there a scripting language that has built in support for threads that are not confined to a single core?

WRAP UP

Answers were not quite what I expected, but the TCL answer comes close. I'd like to add perl, which (much like TCL) has interpreter-based threads.

Jython, IronPython and Groovy fall under the umbrella of combining a proven language with the proven virtual machine of another language. Thanks for your hints in this direction.

I chose Aiden Bell's answer as Accepted Answer. He does not suggest a particular language but his remark was most insightful to me.

Orten

You're question title doesn't make sense when compared to the body of your question. Python's multiprocessing package is multiprocessing, not multithreading. I think you meant to ask "Which scripting languages support multithreading". – Este

This will also depend on what you call a scripting language. Does your definition include PowerShell? What about C#? It can be called from embedded script using Visual Studio Tools for Applications. – Woodcut

@ Dan Lorenc: Thanks, I wasn't aware that multiprocessing is used with different meaning in different contexts. Add clarification to the question. – Orten

@John Saunders: I was deliberately vague on this, because I didn't want another flame war about what makes a scripting language. I'd say take scripting language in the widest sense that is OK for you. – Orten

@Ludwig: if it's ok with you, it's ok with me! I just wanted to point out that one can implement "scripting" through full-blown C# or VB.NET. – Woodcut

Where have you heard that Haskell uses only green threads? There are a lot of articles out about how Haskell works with multicore systems. – Tussore

@Kathy Van Stone: Fair point, I copied the list of language with green threads from en.wikipedia.org/wiki/Green_threads. I have since edited it heavily, also because as stackoverflow.com/users/14143/simon pointed out it contained languages which are not really scripting languages. – Orten

Thank you Ludwig, and nice answer summary ... very useful. and good luck! – Newfashioned

Answer 1

Thread syntax may be static, but implementation across operating systems and virtual machines may change

Your scripting language may use true threading on one OS and fake-threads on another.

If you have performance requirements, it might be worth looking to ensure that the scripted threads fall through to the most beneficial layer in the OS. Userspace threads will be faster, but for largely blocking thread activity kernel threads will be better.

Newfashioned

He said he's not interested in "green threads". So no, userspace threads don't count. – Lazuli

Fair point, but most modern operating systems support kernelspace threads, in contrast none of the modern scripting languages seem to do so. True threads was meant in contrast to processes, in order to exclude answers that point me to pythons multiprocessing module or the like. – Orten

@Ludwig ... This is what I mean. There is a difference between syntax support and real support. Different Python vms may handle the semantics of the same threading syntax in a different way. Same goes for scripting languages on other operating systems. – Newfashioned

@cresentfresh - Acknowledged. Missed that :) – Newfashioned

I don't actually care too much about the colour of my threads;-) Seriously, as I understand it green threads can't run on multiple cores, so they don't count. – Orten

Answer 2

You seem use a definition of "scripting language" that may raise a few eyebrows, and I don't know what that implies about your other requirements.

Anyway, have you considered TCL? It will do what you want, I believe.

Since you are including fairly general purpose languages in your list, I don't know how heavy an implementation is acceptable to you. I'd be surprised if one of the zillion Scheme implementations doesn't to native threads, but off the top of my head, I can only remember the MzScheme used to but I seem to remember support was dropped. Certainly some of the Common LISP implementations do this well. If Embeddable Common Lisp (ECL) does, it might work for you. I don't use it though so I'm not sure what the state of it's threading support is, and this may of course depend on platform.

Update Also, if I recall correctly, GHC Haskell doesn't do quite what you are asking, but may do effectively what you want since, again, as I recall, it will spin of a native thread per core or so and then run its threads across those....

Buonomo

I cleaned up the list with 'scripting' languages, thanks. TCL is a good hint though, neither GIL nor green threads as it seems. – Orten

TCL has it's own approach, but you can certainly get it to use all of your cores, if that's the primary issue. At least on the platforms I've done that on (which wasn't recent). I suspect it may not be in the default build either, but it can be turned on and rebuild easily enough. – Buonomo

Tcl most certainly does spin up native threads; that's pretty much the only reason you'd want to use them in that language anyway (as it has strong async I/O support). They're also robust and have very few shared locks so you get good performance; they've had a lot of production pounding in things like heavy duty webservers... – Rite

Answer 3

You can freely multi-thread with the Python language in implementations such as Jython (on the JVM, as @Reginaldo mention Groovy is) and IronPython (on .NET). For the classical CPython implementation of the Python language, as @Dan's comment mentions, multiprocessing (rather than threading) is the way to freely use as many cores as you have available

Chromogen

JRuby users report vfantastic multi-thread support (and faster that standard Ruby). – Zachar

Answer 4

As Groovy is based on the Java virtual machine, you get support for true threads.

Pamplona

Answer 5

F# on .NET 4 has excellent support for parallel programming and extremely good performance as well as support for .fsx files that are specifically designed for scripting. I do all my scripting using F#.

Irremissible

Answer 6

An answer for this question has already been accepted, but just to add that besides tcl, the only other interpreted scripting language that I know of that supports multithreading and thread-safe programming is Qore.

Qore was designed from the bottom up to support multithreading; every aspect of the language is thread-safe; the language was designed to support SMP scalability and multithreading natively. For example, you can use the background operator to start a new thread or the ThreadPool class to manage a pool of threads. Qore will also throw exceptions with common thread errors so that threading errors (like potential deadlocks or errors with threading APIs like trying to grab a lock that's already held by the current thread) are immediately visible to the programmer.

Qore additionally supports and thread resources; for example, a DatasourcePool allocation is treated as a thread-local resource; if you forget to commit or roll back a transaction before you end your thread, the thread resource handling for the DatasourcePool class will roll back the transaction automatically and throw an exception with user-friendly information about the problem and how it was solved.

Maybe it could be useful for you - an overview of Qore's features is here: Why use Qore?.

Coon

Answer 7

CSScript in combination with Parallel Extensions shouldn't be a bad option. You write your code in pure C# and then run it as a script.

Cabotage

Answer 8

It is not related to the threading mechanism. The problem is that (for example in python) you have to get interpreter instance to run the script. To acquire the interpreter you have to lock it as it is going to keep the reference count and etc and need to avoid concurrent access to this objects. Python uses pthread and they are real threads but when you are working with python objects just one thread is running an others waiting. They call this GIL (Global Interpreter Lock) and it is the main problem that makes real parallelism impossible inside a process.

https://wiki.python.org/moin/GlobalInterpreterLock

The other scripting languages may have kind of the same problem.

Ulm

Answer 9

Guile supports POSIX threads which I believe are hardware threads.

Flowerpot