CruiseControl [.Net] vs TeamCity for continuous integration?

Question

I would like to ask you which automated build environment you consider better, based on practical experience. I'm planning to do some .Net and some Java development, so I would like to have a tool that supports both these platforms.

I've been reading around and found out about CruiseControl.NET, used on stackoverflow development, and TeamCity with its support for build agents on different OS-platforms and based on different programming languages. So, if you have some practical experience on both of those, which one do you prefer and why?

Currently, I'm mostly interested in the ease of use and management of the tool, much less in the fact that CC is open source, and TC is a subject to licensing at some point when you have much projects to run (because, I need it for a small amount of projects).

Also, if there is some other tool that meets the above-mentioned and you believe it's worth a recommendation - feel free to include it in the discussion.

Atul

Answer 1

I have worked on and with Continuous Integration tools since the one that spawned Cruise Control (java version). I've tried almost all of them at some point. I've never been happier than I am with TeamCity. It is very simple to set up and still provides a great deal of power. The build statistics page that shows build times, unit test count, pass rate etc. is very nice. TeamCity's project home page is also very valuable. For simple .NET projects you can just tell TeamCity where the solution is and what assemblies have tests and that is all it needs (other than source control location). We have also used some complicated MSBuild scripts with it and done build chaining. I have also gone through two TeamCity upgrades and they were painless.

CruiseControl.NET also works well. It is trickier to set up but it has a longer history so it is easy to find solutions on the web. Since CruiseControl.NET is open source you also have the option of adding or changing whatever you like. I had used CruiseControl.NET since its release and wrote some of the early code for cc.tray (thankfully re-written by someone who knew better).

Cruise, from ThoughtWorks, also looks quite good but I don't see a compelling reason for me to switch. If I were starting a new project I might give it a try, but TeamCity has done a great job of making the simple things simple while making the complex quite painless.

Edit: We just upgraded to TeamCity 5.0 a few weeks ago and it was another painless upgrade. It let us take advantage of the improved code coverage capabilities and GIT support. We are also now using the personal build and pre-tested commit features that have been in for a while. I just thought I should update the answer to indicate that TeamCity keeps improving and is still easy to use.

Studner

wonder if with all your experience you could have a little bit to share a screencast of creating a simple .NET project on it and work with TeamCity (using JING (free screencasting) or other tool) and post the link here or somewhere so me and all of us struggling with it will have a good startup :) Thank you. – Kurtkurth

@Kurtkurth - I've found the team city docs to be pretty good. And @boj's answere provided a link to a good screen cast here blog.wekeroad.com/kona/kona-2 – Studner

I was always scared of CI because of the image of CCNet that I had, last week I tried TeamCity 6.5 and it has been an absolute joy. didn't touch a single xml/configuration file. – Idiophone

If you're currently using CC.Net and you'd like to move to Team City, is it painful? How does TC produce MSI files? – Keifer

@Wes, I don't know if this is still relevant to you, but we have 20+ projects running in CC.Net, but recently I implemented TeamCity for some Android projects we have. The greatest pain by far was learning Ant, but that's something I would have to have done if I had put the projects in CC.Net. I would happily move all our projects into TeamCity if I could convince management to pay for the extra build configurations we would need (it's free to a point), mainly due to configuration, which is a breeze in TeamCity and an XML quagmire in CC.Net. – Mariannemariano

Answer 2

I was/I'm a big fan of CC.NET. We have currently 5 projects in CruiseControl, and works great. Writing config files with hand can be painfull but it's okay.

But.

After the Kona: Continuous Integration and Better Unit Testing screencast (the first 1/3 about TeamCity) I'll check TeamCity too. I love the integrated unit test dashboard and the configuration interface.

I think everybody should watch this video before choosing CC.NET or TeamCity.

p.s.: I hope there is a valuable CC.NET video on the net too.

Blinkers

Answer 3

My favorite CI server by far is Hudson. Easy to set up and maintain, lots of nice graphs for showing trends to developers and non-developers, and free.

I am using TeamCity currently on a project and I'm generally pleased with it, but many of the graphs it generates aren't especially useful, and it is more complicated to configure than Hudson.

That said, TeamCity is powerful, free for many uses, and has one killer feature: Remote Run. You can "pre-commit" your check in straight from IDEA or Eclipse, run one or more build configurations on the TeamCity server, and only commit the changes if the build is successful (e.g., compiles and all tests pass).

Given that you could get both TeamCity and Hudson up and running in a few hours, it might be worth grabbing both and running them side-by-side, along with any others (such as CruiseControl) that you can think of. If you can't stand a CI server up quickly to do a side-by-side comparison, then at least you have a data point for easy of install and/or configuration.

Counter

+1 for Hudson. See also: using Hudson as a .NET build tool: #616649; CruiseControl vs Hudson: #604885 – Incommodity

Answer 4

I have used them both successfully on different projects. From a setup and administrative standpoint Team City is far easier to deal with. You don't have to hack around with .config files like you do with CC and setup is a breeze. Since you do not have a lot of projects I would recommend Team City over CC until you get to the point that Team City costs $$.

Che

Answer 5

I have used both CC.net and TeamCity. I am tasked with setting up and installing TeamCity for my organization (5 developers). Our organization uses some uncommon practices and tools (at least, for orgs of our size), such as Perforce for source control and multiple build agents running on heterogeneous operating systems, which caused some initial setup headaches. However, the support via email was absolutely top-notch in getting everything set up. I received answers to my dumb questions in literally minutes.

The interface is intuitive and responsive, as well as feature-packed. The product feels very expensive. Configuration is easy, and the web interface is intellegent enough to update itself without any restarting of the agent or server services, or even refreshing of the page.

I feel like we're using just about every advanced feature of the product and have found no bugs at all so far. Ndepend integration, nested NAnt scripts, Perforce version labeling, you name it, we're doing it.

I highly recommend TeamCity to anyone looking for a continuous integration server, or any build server, really.

Varix

Perforce is an uncommen tool? Perforce is fantastic for teams of any size. I use it myself for my home-development :) – Cnemis

@sonstabo, I would agree with that, most small companies go with SVN or GIT and Perforce is mostly used at larger companies. I've used Perforce and liked it but I was at a 1,000 developer company when I used it. – Pyroxenite

Answer 6

Without wanting to throw alternative tools at you :-)

Hudson is a great open source alternative, I have used CC and CC.net, and I confess i do think they are fantastic tools. I am pondering switching to hudson as it apears a lot easier to setup and maintain.

https://hudson.dev.java.net/

Abecedarium

+1 As no one seems to think about Hudson for .Net CI. I've used Cruise Control, Team City, and Bamboo, and found Hudson to be the best tool for the job in most cases where budget is a big concern. – Belia

Answer 7

Make sure the system that you decide on scales to the number of projects that you'll need it to handle...

I use CruiseControl.Net but I wouldn't recommend it for building lots of projects... I have a (possibly slightly strange) arrangement where I have many C++ static libraries which I compose into applications. Each library depends on other libraries and the apps pull in a set of libs and build. Each lib has a test suite. Each app has a test suite. I build for 5 compilers and variations of (windows) platforms.

The first thing I found was that CC.Net's project triggers aren't really quite what you need and the multi-trigger doesn't play well with project triggers. The way project triggers work (they use remoting to connect to the server where the project is stored (even if it's a project that is managed by the same instance of CC.Net) and then pull all projects from that server and search the list sequentially looking for the project that you're interested in...) means that they don't scale well. Once you get above a certain number of projects you'll find that CC.Net is taking most of the CPU for your build machine.

Of course, it's open source, so you can fix it... And, I'm sure it's fine for small numbers of non-interdependent projects.

For more details of the problems I had and some patches for CC.Net see here http://www.lenholgate.com/archives/cat_ccnet.html

Togs

oh... thanks for this info, valuable post. especially when said that TC is a simple setup and its limitation is number of project, where the CC & CC.Net might come into play. but as you say, it gets into problems when project count rises... thanks again – Atul

Even in my heavily hacked version of CC.Net I have problems because CC.Net uses a thread per project approach to scheduling and, well, that's just a crap design IMHO. – Togs

Answer 8

I have recently setup cc .net. It is a great application but does require a bit of patience. You will be editing config files in notepad alot :)

It has been around a while so its well supported and you can normally find someone who has done what you wantto do before. The web interface is .net as well which was a plus for us as we are a Microsoft shop.

I havent used TeamCity but I have heard quite a few recommendations of it and it looks pretty.

Badlands

Answer 9

I had an experience setting up and running CruiseControl (Java version) on Linux during my previous company. Like most people suggest, it is not the most trivial thing to setup. You need to understand its framework in order to come up with the workable/managable config. However, once you passed that hump, I feel that CruiseControl is pretty flexible enough to allow you to do different sort of things to fit different scenarios.

Besides, CruiseControl documentation, its wiki page also has some useful information as well.

I do not have a direct experience with TeamCity. Though its pre-test commit feature looks interesting enough.

The other CC tool that you might give it a look is Bamboo from Atlassian. It is a lot more easier to setup and the interface is nicer. Though, it is not as flexible as what CruiseControl offers.

Convoke

Answer 10

A third option you might want to consider: Thoughtworks' Cruise. It's built on CruiseControl, but offers a lot more features, easier setup, etc, etc. Not free (or open source).

http://studios.thoughtworks.com/cruise-continuous-integration

Timorous

I have looked into Cruise a few times when it comes to setting up as a CI system. it's not much simpler than for example CC.NET. It has its strengths in build and deploy flows but as a CI tool i thing TeamCity is a lot more competent – Immoderacy

If you're going to pay for Cruise/Go, you may as well get TeamCity for free. – Staffer

Answer 11

I have been using Teamcity for the last 1 and a half years, and having a great experience. I have integrated a number of .Net and Java projects and used tools like MSBuild, Maven etc. I found Teamcity pretty simple to set up and work with. I have managed to get CI running for some sql projects as well which was a bit of nightmare which could have been worse with other CI tools.
Recently upgraded to Teamcity 8.0.6 which was painless. Also Teamcity provides a REST API which is very useful for some scenarios. If you are using powershell for automating builds a number of Psake/Teamcity integration scripts are available on GitHub

Ignatius