All the tests passed, but bamboo build fails with a statement "No failed tests found, a possible compilation error occurred."

Question

I'm supposed to run some jbehave(automated) tests in bamboo. Once the tests run I'll generate some junit compatible xml files so that bamboo could understand the same. All the jbehave tests are ran as part of a script, because I need to run the jbehave tests in a separate display screen(remember these are automated browser tests). Example script is as follows.

Ex:

export DISPLAY=:0 && xvfb-run --server-args="-screen 0, 1024x768x24" 
mvn clean integration-test -DskipTests -P integration-test -Dtest=*

I have one more junit parser task which points to the generated junit compatible xml files. So, once the bamboo build runs and even if all the tests pass, I get red build with the message "No failed tests found, a possible compilation error occurred."

Can somone please help me on this regard.

Winterize

Answer 1

Your build script might be producing successful test reports, but one (or both, possibly) of your tasks is failing. That means that the failure is probably* occurring after your tests complete. Check your build logs for errors. You might also try logging in to your Bamboo server (as the bamboo user) and running the commands by hand.

I've seen this message in the past when our test task was crashing halfway through the test run, resulting in one malformed report that Bamboo ignored and a bunch of successful reports.

*Check the build log to make sure that your tests are indeed running. If mvn clean doesn't clean out the test report directory, Bamboo might just be parsing stale test reports.

---

EDIT: (in response to Kishore's links)

It looks like your task to kill Xvfb is what is causing the build to fail.

18-Jul-2012 09:50:18    Starting task 'Kill Xvfb' of type 'com.atlassian.bamboo.plugins.scripttask:task.builder.script'

18-Jul-2012 09:50:18    
Beginning to execute external process for build 'Functional Tests - Application Release Test - Default Job'
 ... running command line: 
/bin/sh
  /tmp/FUNC-APPTEST-JOB1-91-ScriptBuildTask-4153769009554485085.sh
 ... in: /opt/bamboo-home/xml-data/build-dir/FUNC-APPTEST-JOB1
 ... using extra environment variables: 

<..snip (no meaningful output)..>

18-Jul-2012 09:50:18    Failing task since return code was 1 while expected 0

18-Jul-2012 09:50:18    Finished task 'Kill Xvfb'

What does your "Kill Xvfb" script do? Are you trying something like pkill -f "[x]vfb"? pkill -f silently returns non-zero if it can't match the expression to any processes.

Alix

Choover, thanks for the reply. I don't see any possible errors from any of my bamboo tasks. There are definitely some stack traces which are caused from my application. Please have a look at the buil ci.openmrs.org/browse/FUNC-APPTEST/latest and log ci.openmrs.org/download/FUNC-APPTEST-JOB1/build_logs/… for more details. – Winterize

sorry, wasn't enough space in comment box to post log output, so I ended up editing my post--see above ^ – Alix

Answer 2

My solution was to make a 'script' task:

#!/bin/bash
/usr/local/bin/phpcs --report=checkstyle --report-file=build/logs/checkstyle.xml --standard=PSR2 ./lib | exit 0

Which always exits with status 0.

This is because PHP code sniffer return exit status 1 when only 1 coding violation (warning / error) is found which causes the built to fail.

Longcloth

The same solution worked for me too on our Windows build server. Must use a pipe for the exit rather than a new line, otherwise exit 0 won't -always- be called after running karma! – Gottlieb

Answer 3

Turns out to be a simple fix.

General bamboo behavior is to scan the entire log and see for any failure codes(1). For this specific configuration i had some 6 scripts out of which one of them was to kill the xvfb(frame buffer). For some reason server is not able to kill xvfb and that task was returning a failure code. Because of this, though all the tests passed, bamboo got one of this error codes from previous tasks and build was failing.

Current fix is to remove the task which kills xvfb and the build went green! \o/.

Winterize

Wow, that's exactly what I put in my response to your comment. sigh – Alix

Cool! thanks a lot for taking out some time and looking out for log output :) – Winterize

that was exactly what @Alix put in his response to your comment. tsc tsc tsc You shall sign that response as correct answer. – Gerbold