Leakspin, I'll try to give you some context on whether it's doable, and what has been done yet.
if FFOS uses the kernel from Android, then it can load the drivers from my Android ROM, right?
Yes and No. There is much more than that. As you can see, Firefox is built putting together several pieces, such as Gonk(what you were referring to "android kernel", Gecko (our application runtime), and finally Gaia (user interface). Take a look at the Stack. For more info, please follow the architecture blog post at MDN.
Has it been ported yet to any Android devices?
- Yes, but it's not a simple process and there is no guarantee that it will work on all android devices. Here, have a look at the list of devices and which tools you will need to perform such operation.
There is also a porting tutorial you might want to follow.
f I build my FFOS ROM correctly, would I still have some bugs from hardware/drivers?
- Unfortunately, very likely that you will do.
I tried my best to style this answer, but the stupid stackoverflow limited the amount of links and images I'm able to post. Sorry for that.
Links:
How to port Firefox OS
Pre-requisites.