Cruise Control .NET time build spends in failed state

Question

My team has a goal to minimize the amount of time that our build is broken.

We use CruiseControl.NET for continuous integration. What I'd like to find out is how best to approach answering the following question:

"In the last {timespan}, how much time has {project-name} spent in a broken status?"

For example: "Over the last 1 month, how much time has our project spent in a broken status?"

Are there any advanced features of CruiseControl.NET that would facilitate making this information available in some type of a report or somewhere in the dashboard?

Alternatively, how would you approach parsing the xml artifact files to glean this info?

Brambly

"My team has a goal to minimize the amount of time that our build is broken." Why you need this? Please describe a scenario when you can use this information. If you have 10h in broken status or 3 days, what it will give you? – Nival

Restuta: For our team, a broken build represents a bug. We have a suite of automated tests at various levels of our system. When one of these tests fails due to a code change, it means we've added a bug. Bugs are not tolerable and we would like to enforce fixing them as a high priority. Having the statistics is just a means to quantify our performance on meeting our goal of quickly attending to and fixing bugs. Also, we use CruiseControl.NET to deploy our dev site, so as long as a build is broken, new code changes won't be reflected on our dev site. We'd like to keep our dev site up to date. – Brambly

Broken build count is IMHO a much better metric than the time spent in this state. – Gatt

Upvoted. I think it's a valid and interesting question. These sort of metrics are fun, especially if taken lightly (ie, people want to improve this metric because they are perfectionists and light-heartedly competetive, rather than because their HR manager is an idiot and their Bonus will be affected). Also consuming the Cruise Control API (or scraping or whatever the answer turns out to be) will be interesting. – Shoestring

I agree that broken build count is a good metric, but it has different implications than time spent broken. Broken build count tells us how often we introduce defects. Time spent in a broken build state tells us how quickly we respond to those defects. – Brambly

Answer 1

I see at least two ways to approach this:

1. You write an external tool which parses CC.NET's XML log files for a project (stored in buildlogs subdirectory by default), calculates statistics and writes a HTML report. This is probably easier to do, but it won't be directly integrated with CC.NET.
2. You write a CC.NET plug-in to do this. You'll need to do a bit of investigating in this case. My guess the starting point would be to look at the source code of some existing plug-in.

Here are some links about CCNET plugins:

• http://www.cruisecontrolnet.org/projects/ccnet/wiki/DevInfo_MakingPlugins
• BrekiLabeller - my own plug-in, useful if you want to see how a plug-in can be implemented.

Thousandfold

Answer 2

you can use the statistics publisher, http://www.cruisecontrolnet.org/projects/ccnet/wiki/Statistics_Publisher and you can display them via project statistics plugin

Boisleduc

Answer 3

Having had a very quick look at the CC docs, I imagine if you were writing your own Cruise control dashboard, you could consume the RSS feed of build results, parse in all the date times and success/failure states up to your threshold, then sum up the totals.

As for displaying it in a dashboard, I think Cruise Control has a plugin architecture which might help http://cruisecontrol.sourceforge.net/main/plugins.html

Shoestring

I'm referring specifically to CruiseControl.NET. Do you know if any of that documentation from CruiseControl also applies to CruiseControl.NET? – Brambly

Answer 4

So my eventual solution wasn't ideal, but it was easy to do and it works:

I had CC.NET send build emails to an email address (we'll call it build_emails@build_statistics.com). Then I use a ruby script to get the emails via imap and process them to determine our build failure time.

I didn't go the route of directly parsing the xml because I would have had to parse every xml file in the timeframe to build up a timeline and then go over the timeline to make my calculations. It just seemed too complicated to get a simple statistic like this.

Brambly

Update: this turned out to be a terrible unmaintainable solution. :P – Brambly

Answer 5

I like cc.net, but in this case TeamCity just does this for you. It has lots of other great statistics too. It's free for less than 20 projects.

Ashleighashlen

Team City does look pretty neat. Out of curiosity, how would one answer this question using TeamCity? – Brambly

@Brambly - Team City shows statistics for each build configuration that shows the time to fix the build over a day, week, month, quarter, year or all time. their docs show an example of the statistics page confluence.jetbrains.net/display/TCD5/Statistics. It also has a restful api that you might be able to use to query it. Can't say I have tried the query method yet. I just look at the graph. – Ashleighashlen