CloudFoundry/Springboot app using Liberty for Java on Bluemix all of a sudden crashes when starting, no changes in code for months

T

2

7

I have a javascript/html/css application being served with Springboot on Bluemix. Running the app locally works just fine, and the code has not been changed in a couple months. When I attempt to push the app using the CF CLI, I get the following error:

   2017-11-13T12:18:29.89-0600 [CELL/0] OUT Successfully destroyed container
   2017-11-13T12:19:33.32-0600 [CELL/0] OUT Creating container
   2017-11-13T12:19:37.70-0600 [CELL/0] OUT Successfully created container
   2017-11-13T12:19:55.79-0600 [CELL/0] OUT Starting health monitoring of container
   2017-11-13T12:19:59.83-0600 [APP/PROC/WEB/0] OUT 18:19:59.812 [main] INFO app.Application -  App Started 
   2017-11-13T12:20:03.49-0600 [APP/PROC/WEB/0] OUT 2017-11-13 18:20:03,485 INFO  [main] app : app.Application : Starting Application on  with PID 13 (/home/vcap/app/BOOT-INF/classes started by vcap in /home/vcap/app)
   2017-11-13T12:20:03.49-0600 [APP/PROC/WEB/0] OUT 2017-11-13 18:20:03,487 INFO  [main] app : app.Application : No active profile set, falling back to default profiles: default
   2017-11-13T12:20:03.79-0600 [APP/PROC/WEB/0] ERR J9Generic_Signal_Number=00000004 Signal_Number=0000000b Error_Value=00000000 Signal_Code=00000001
   2017-11-13T12:20:03.79-0600 [APP/PROC/WEB/0] ERR RCX=0000000000000536 RDX=0000000000000000 R8=000000000000000C R9=00000000FFFFFFF9
   2017-11-13T12:20:03.79-0600 [APP/PROC/WEB/0] ERR RIP=00007F82CB50AADF GS=0000 FS=0000 RSP=00007F82CC39F460
   2017-11-13T12:20:03.79-0600 [APP/PROC/WEB/0] ERR xmm12 4207fffffff8016a (f: 4294443264.000000, d: 1.288490e+10)
   2017-11-13T12:20:03.79-0600 [APP/PROC/WEB/0] ERR xmm13 3c57c00000000000 (f: 0.000000, d: 5.149960e-18)
   2017-11-13T12:20:03.79-0600 [APP/PROC/WEB/0] ERR Module=/home/vcap/app/.java/jre/lib/amd64/compressedrefs/libj9vm29.so
   2017-11-13T12:20:03.79-0600 [APP/PROC/WEB/0] ERR CPU=amd64 (4 logical CPUs) (0x7daa33000 RAM)
   2017-11-13T12:20:03.79-0600 [APP/PROC/WEB/0] ERR (0x00007F82CB4E869F [libj9vm29.so+0xeb69f])
   2017-11-13T12:20:03.79-0600 [APP/PROC/WEB/0] ERR (0x00007F82CB4DED50 [libj9vm29.so+0xe1d50])
   2017-11-13T12:20:03.79-0600 [APP/PROC/WEB/0] ERR Java_java_lang_ClassLoader_defineClassImpl+0xa4 (0x00007F82C0D69134 [libjclse7b_29.so+0x11134])
   2017-11-13T12:20:03.79-0600 [APP/PROC/WEB/0] ERR Unhandled exception
   2017-11-13T12:20:03.79-0600 [APP/PROC/WEB/0] ERR Type=Segmentation error vmState=0x00000000
   2017-11-13T12:20:03.79-0600 [APP/PROC/WEB/0] ERR Handler1=00007F82CB481AA0 Handler2=00007F82CAD8BE00 InaccessibleAddress=0000000000000001
   2017-11-13T12:20:03.79-0600 [APP/PROC/WEB/0] ERR EFlags=0000000000010246 CS=0033 RBP=00007F82CC39F9B0 ERR=0000000000000004
   2017-11-13T12:20:03.79-0600 [APP/PROC/WEB/0] ERR TRAPNO=000000000000000E OLDMASK=0000000000000000 CR2=0000000000000001
   2017-11-13T12:20:03.79-0600 [APP/PROC/WEB/0] ERR xmm6 1610000ab3f109b8 (f: 3018918400.000000, d: 2.041302e-202)
   2017-11-13T12:20:03.79-0600 [APP/PROC/WEB/0] ERR xmm11 3c4cd58858eb8000 (f: 1491828736.000000, d: 3.126201e-18)
   2017-11-13T12:20:03.79-0600 [APP/PROC/WEB/0] ERR xmm15 4028f40b5ed97f00 (f: 1591312128.000000, d: 1.247665e+01)
   2017-11-13T12:20:03.79-0600 [APP/PROC/WEB/0] ERR (0x00007F82CB4DDEDA [libj9vm29.so+0xe0eda])
   2017-11-13T12:20:03.79-0600 [APP/PROC/WEB/0] ERR R14=0000000000000000 R15=0000000000000000
   2017-11-13T12:20:03.79-0600 [APP/PROC/WEB/0] ERR xmm0 00ff000000000000 (f: 0.000000, d: 7.063274e-304)
   2017-11-13T12:20:03.79-0600 [APP/PROC/WEB/0] ERR xmm2 6a003e74696e693c (f: 1768843520.000000, d: 3.978864e+202)
   2017-11-13T12:20:03.79-0600 [APP/PROC/WEB/0] ERR xmm7 b8d41014f309b3f1 (f: 4077499392.000000, d: -6.037435e-35)
   2017-11-13T12:20:03.79-0600 [APP/PROC/WEB/0] ERR xmm10 3fe0000000000000 (f: 0.000000, d: 5.000000e-01)
   2017-11-13T12:20:03.79-0600 [APP/PROC/WEB/0] ERR ----------- Stack Backtrace -----------
   2017-11-13T12:20:03.79-0600 [APP/PROC/WEB/0] ERR (0x00007F82CB4E6C85 [libj9vm29.so+0xe9c85])
   2017-11-13T12:20:03.79-0600 [APP/PROC/WEB/0] ERR ---------------------------------------
   2017-11-13T12:20:03.79-0600 [APP/PROC/WEB/0] ERR R10=0000000000000522 R11=00007F82CC39FA0C R12=0000000000000000 R13=00007F82CBB9F520
   2017-11-13T12:20:03.79-0600 [APP/PROC/WEB/0] ERR xmm3 105953dd10130610 (f: 269682176.000000, d: 6.525551e-230)
   2017-11-13T12:20:03.79-0600 [APP/PROC/WEB/0] ERR xmm4 d91013075953130a (f: 1498616576.000000, d: -1.037698e+121)
   2017-11-13T12:20:03.79-0600 [APP/PROC/WEB/0] ERR xmm5 5953cc0913045953 (f: 319052128.000000, d: 2.044833e+122)
   2017-11-13T12:20:03.79-0600 [APP/PROC/WEB/0] ERR xmm8 0000000000000000 (f: 0.000000, d: 0.000000e+00)
   2017-11-13T12:20:03.79-0600 [APP/PROC/WEB/0] ERR xmm9 09b3fe01b8042e01 (f: 3087281664.000000, d: 6.348990e-262)
   2017-11-13T12:20:03.79-0600 [APP/PROC/WEB/0] ERR Module_base_address=00007F82CB3FD000
   2017-11-13T12:20:03.79-0600 [APP/PROC/WEB/0] ERR Target=2_90_20170901_363591 (Linux 4.4.0-75-generic)
   2017-11-13T12:20:03.79-0600 [APP/PROC/WEB/0] ERR (0x00007F82CB50AADF [libj9vm29.so+0x10dadf])
   2017-11-13T12:20:03.79-0600 [APP/PROC/WEB/0] ERR (0x00007F82CB502134 [libj9vm29.so+0x105134])
   2017-11-13T12:20:03.79-0600 [APP/PROC/WEB/0] ERR (0x00007F82CB4E7E64 [libj9vm29.so+0xeae64])
   2017-11-13T12:20:03.79-0600 [APP/PROC/WEB/0] ERR (0x00007F82C0D74DD9 [libjclse7b_29.so+0x1cdd9])
   2017-11-13T12:20:03.79-0600 [APP/PROC/WEB/0] ERR RDI=00007F82CC39F9B0 RSI=0000000000000000 RAX=0000000000000001 RBX=0000000000000001
   2017-11-13T12:20:03.79-0600 [APP/PROC/WEB/0] ERR xmm1 0000ffffffffffff (f: 4294967296.000000, d: 1.390671e-309)
   2017-11-13T12:20:03.79-0600 [APP/PROC/WEB/0] ERR xmm14 bc5b599f227becbb (f: 578546880.000000, d: -5.930604e-18)
   2017-11-13T12:20:03.79-0600 [APP/PROC/WEB/0] ERR (0x00007F82B053920E [<unknown>+0x0])
   2017-11-13T12:20:03.89-0600 [APP/PROC/WEB/0] OUT Exit status 255
   2017-11-13T12:20:03.92-0600 [CELL/0] OUT Exit status 0
   2017-11-13T12:20:03.94-0600 [CELL/0] OUT Destroying container
   2017-11-13T12:20:03.94-0600 [CELL/0] OUT Stopping instance 5a3e2441-bbd9-402c-5748-fd68
   2017-11-13T12:20:03.96-0600 [API/0] OUT Process has crashed with type: "web"
   2017-11-13T12:20:03.97-0600 [API/0] OUT App instance exited with guid f7a4a8ff-ebbc-49cf-96eb-37895fb88edd payload: {"instance"=>"5a3e2441-bbd9-402c-5748-fd68", "index"=>0, "reason"=>"CRASHED", "exit_description"=>"APP/PROC/WEB: Exited with status 255", "crash_count"=>4, "crash_timestamp"=>1510597203938616258, "version"=>"7ebb8072-9edc-40e3-a749-ce83273fb0d3"}

I have other apps using the liberty for java buildpack on Bluemix that are doing fine, but this app specifically has started crashing with no changes to the environment or code, and the errors are not too helpful. I've tried increasing memory and using several different buildpacks (IBM's websphere libery as well as cloud foundry's) to no avail Any advice would be appreciated.

Tacho answered 13/11, 2017 at 18:30 Comment(0)

T

3

Was able to fix this by using the community java buildpack, java_buildpack

It seems that there was a problem with the default buildpack. I imagine an update to the buildpack caused something to break.

Tacho answered 13/11, 2017 at 20:34 Comment(2)

When you use the java-buildpack the application is executed as a java app (in this case the Springboot with embedded container). It is not running on a Liberty container. – Barbarity 15/11, 2017 at 22:53

I opened a support request about the same issue. IBM guys told me: "For IBM Buildpacks, you cannot deploy a JAR file to Bluemix. You can only deploy WAR's, EAR's, server directories and packaged servers." ... Anyway, I don't use anymore their buildpacks. I use the community java buildpack as explained by @treevez: cf push -b java_buildpack – Unilateral 14/12, 2017 at 7:40

D

2

Have you tried increasing the memory for the apps and try the liberty buildpack?

Didymous answered 13/11, 2017 at 18:33 Comment(1)

Yes, sorry I just edited right after posting to include that info. – Tacho 13/11, 2017 at 18:34

Recommended topics

Hot tags