What is the difficulty in making Smalltalk parallel?

Question

Given the core model of objects that are independent computing engines (storage being instance vars, CPU being the class methods) responding to messages passed from one to the other, it would seem that Smalltalk would be a natural fit for parallel processing on massive numbers of cores. Yet this is an area where Smalltalk is still very weak indeed, replying on its own simulated multitasking features which do not exploit modern processors' hardware capabilities.

Why is this? What are the main problems? Is mutability the key, or is it something more specific to Smalltalk?

Pappas

Answer 1

First of all, let's recall that GemStone proves that Smalltalk can be extended to support parallel computing. Of course, the question remains because GemStone is a very sophisticated system (think, e.g., in the Garbage Collector!) and all other dialects don't behave that way.

Parallel computing requires thread-safe code; otherwise race conditions would emerge all the time. The problem is that making Smalltalk code thread-safe would add complexity everywhere. Consider for instance

OrderedCollection >> addLast: anObject
  lastIndex = array size ifTrue: [self makeRoomAtLast].
  lastIndex := lastIndex + 1.
  ^array at: lastIndex put: newObject

An interruption between these lines of code could leave the internal state of the receiver inconsistent. Of course, this could also happen with non-parallel execution because Smalltalk supports interruptions. However, the way Smalltalk uses this capability is limited to critical sections that do not happen so often and are therefore somehow under control.

The reason why the addition of thread-safe code is not so natural in Smalltalk is that in Smalltalk we have an image. This means that there are lots of objects shared by all processes, including for example, all the classes, compiled methods, etc. The dynamic nature of Smalltalk, which is used extensively by the system and its applications, makes things even harder.

In my personal experience a good way to achieve multi-core capabilities in Smalltalk is to launch different OS processes (instances of a headless image) and coordinate them using Smalltalk semaphores and processes. Under this approach, every OS process is represented in the main image (the one with a UI) by a Smalltalk process. The communication between the main image and the headless processes can rely on sockets (or any other feature). Of course, you pay a price when debugging. In fact, you end up tracing lots of events in log files and opening debuggers in the "headless" processes until you understand what went wrong. But can be done, not just as a demo but for a real "industrial-strong" product.

Serous

Yes of course an "interruption" in a critical place would destroy properties expected of code. But this isn't peculiar to SmallTalk; every parallel language has this problem. They solve by insisting that a) you can't interrupt some things) or b) some things are transactional and complete no matter what. So the key problem with "making Smalltalk parallel" are the addition of parallel threads, and the addition of transactional safety (both on the part of what the execution engine does) and where the programmer insists he needs it. – Premature

@IraBaxter Absolutely. I just wanted to point out that a small amount of parallelism is fine and that the problem comes when you want to take that much further. – Serous

Answer 2

Difficulty ? None

Just launch multiple Pharo instances and have them communicate via sockets or save their final data to a common file. Your OS will manage each instance and send it to execute in a diffirent core. OSProcess module offers such functionality and have been successful implementations like Hydra and RoarVM , the problem is none use them.

Actually the hardest thing of parallelism is to get people using it. With todays hardware applications rarely hit the 100% of a single core. I have barely made Pharo go above 10%.

Also like many programming dynamic programming languages Smalltalk is a developer performance language and not a application performance language.

If you really have such a heavy processing problem you should be using languages like C and C++ that are very application performance orientated languages. Not only its way harder to use those language but even parallelism is very hard to do right even with the right library. The hardware is very weird design wise and there like a ton of gotchas you have to be aware of.

And this is why parallelism suits those programming languages better. Of course you could make libraries in C/C++ and have Pharo or other smalltalks use them. Python does this. Python and Pharo are quite similar in that both they use a GIL and have green threads. The gotcha is that you will have to join your threads back to the main thread in order for the VM to have direct access to it but there are ways around this like I said, socket communication, pipes, Shared memory mapped files and much more.

Python's parallel libraries are all C/C++ based.

Parallelism itself is a very tricky subject , even if you have parallelism its possible that your code will be as slow as running on a single thread and single core. But that is a general problem with application performance , the moment you want to milk out as much power as you can you have to know how hardware works.

Hardware itself nowdays is super complex. Language is the least of your concerns.

So totally possible in Smalltalk but frankly there are not that many people interested into it. I have seen questions on parallelism in the Pharo mailing lists that I frequent that last 2 years, maybe once or twice . And even for concurrency its very rare for someone to ask a question about it.

Eliath

If only someone could come up with a solution in code, accepted by the community and working well and put that into the usual dialect's images, while hiding the complexities of having to build the services which accept work from other running images (or stealing work) and return the result … That would be more exciting than to download Pharo or Squeak having the issue when wanting to use multiple cores. I don't doubt that Smalltalk programmers can code their own. What about beginners? "You can do it with multiple images." metatalk seems not sufficiently motivational. Have tutorials? – Enrika

Pharo instances and have them communicate via sockets or save their final data to a common file. You have to be crazy to think that this is a palatable approach to multithreading. – Adulteration

The lack of questions on the mailing lists may just be a result of the clear answers (like this one) that come up when you search Google for the question. – Rightminded

@ThrowAwayAccount This isn't an answer, it's an excuse. And a ridiculous one at that. – Pappas

The big place you hit the need for true, native threads is web applications. With true, native threads, you can take maximum advantage of multiple CPU cores. – Flabby