scala filename too long

Question

I'm using scala 2.10 and gradle 1.11

My problem is that the compiled jar drop an error when I try to running in the hadoop cluster. I want to run on hadoop because I using scalding.

The exception is:

Exception in thread "main" java.io.FileNotFoundException:
/tmp/hadoop-root/hadoop-unjar6538587701808097105/com/twitter/bijec
tion/GeneratedTupleCollectionInjections$$anon$31$$anonfun$invert$10$$anon
fun$apply$46$$anonfun$apply$47$$anonfun$apply$48$$anonfun$apply$49$$anonfu
n$apply$50$$anonfun$apply$51$$anonfun$apply$52$$anonfun$apply$53$$anonfun$app
ly$54$$anonfun$apply$55.class (File name too long)

Any comments are welcome...

Romaineromains

Are you running on an encrypted filesystem? It's bit me before and many others as well: groups.google.com/forum/#!topic/simple-build-tool/wtD6vgdiy6g – Botel

nope I running in a docker container – Romaineromains

Docker only supports up to 242 byte long file names (github.com/dotcloud/docker/issues/1413) and in your case the auto-generated file name is 252 bytes long. – Botel

Is there any solution for this? Maybe I can use maven shade plugin to rename the classes but its too ugly solution... – Romaineromains

See my answer below. It should solve your problem. – Botel

Answer 1

Adding -Xmax-classfile-name 200 to the scalac options should fix that.

Sources:

https://issues.scala-lang.org/browse/SI-3623

https://groups.google.com/forum/#!topic/simple-build-tool/wtD6vgdiy6g

Botel

but hasn't the problem been because of scalding internal class which he has no control while compiling his own sources? after all the error was about scalding class: com/twitter/bijec tion/GeneratedTupleCollectionInjections... – Pinkham

Agree with what @Pinkham said. The scalac options just works for your own classes having shorter file names. – Brigantine

To expand on this a bit, you want to reduce the max-classfile-name value so that you can sneak under the hard 255 full file path size limit in Linux. – Tarr