What is the main cause of "self-suppression not permitted" in Spark?

Question

When the spark was writing a large file to HDFS using saveAsTextFile, I got an error: java.lang.IllegalArgumentException: Self-suppression not permitted at java.lang.Throwable.addSuppressed(Throwable.java:1043) Caused by: java.io.IOException: All datanodes DatanodeInfoWithStorage

I've searched for a solution to this, but I've not found the right solution and I don't know why.

What is causing these errors, and how can I fix them?

Ophthalmoscope

Possible duplicate of Spark: Self-suppression not permitted when writing big file to HDFS – Mirtamirth

Answer 1

The error self-suppression not permitted is not the actual error here.

This error is thrown by JVM when runtime tries to throw multiple THROWABLE instance from code.

To do so, JDK-7 introduces the "suppressedExceptions" to Throwable to suppress previous exception and retain very recent exception.

So Java tries - throwable.addSuppressed(throwable) - which is invalid and Throwable instance itself throws the IllegalArgumentException and the real exception is lost this way.

---

From this in log: Caused by: java.io.IOException: All datanodes DatanodeInfoWithStorag, It seems like there is some issue with datanode.

If you look into the running services in the cluster, and check if all datanodes and daemons are running fine; things may go in track.

Issues could be:

1. Issue in datanode. Restart cluster daemons.
2. Heart beat between datanodes and namenode: Network Issues or permission access issues.
3. datanode's memory issue.
4. Datanode busy or unresponsive

Appendectomy

The actual error could be hidden behind this weird 'self-supression' error. When you don't see any clue in the yarn logs, check the Spark UI once. You will have some clue on the stage failures there. It would more likely be some memory spill or something similar – Archie

Answer 2

If you look at the documentation of Throwable you'll find some useful information about the addSuppressed method. Specifically, if you have an exception e, and you do e.addSuppressed(e), then a new IllegalArgumentException exception would be generated. This exception would show as

java.lang.IllegalArgumentException: Self-suppression not permitted

And of course, nothing about the original exception e would be shown.

Here are the errors that can be thrown when calling addSuppressed:

Throws: IllegalArgumentException - if exception is this throwable; a throwable cannot suppress itself. NullPointerException - if exception is null

Saprophagous