At our office, we have been using NHibernate for 3 years now. We've been thinking about making a move to Fluent Nhibernate but never made the move. Using hbm.xml files was still the easiest to debug/alter. Two common problems of those xml files are that they are all validated during the creating of the sessionfactory and they are not refactor-safe.
Due to a bug I had to update a newer release of NHibernate (we were using NHib 2.1.2GA) and when I implemented 3.2GA we also were handed the ability to use loquacious mappings (mapping by code). I decided to use Loquacious over Fluent because I don't have a dependency to another project (Fluent) and the fact that NHibernate won't be shipped if mapping by code is broken.
Be aware though, that Loquacious mapping isn't complete either. While I was mapping everything by code, I found out basic stuff like property-ref wasn't always implemented. Thus even though it's shipped, it's not 100% complete. And while this will not come as a shocker, it has bugs. yes. really. ;-)
for more info on (reported) bugs, check out the NHibernate bug database: https://nhibernate.jira.com/browse/NH
Hope this helps.
Regards, Ted