Removing background noise is very difficult. If it is uniform noise, such as the "white" noise on an airplane, there are algorithms to remove those frequency components. But if the noise is non-uniform and similar in nature as your good recording, e.g. people talking in the background, street noise, etc. it will be very difficult to separate them. There has been done a lot of research on this.
At the very least, you'll need to know exactly what the good audio is, and what is the bad audio. E.g. if you're capturing voice, there might be special audio repair software available to separate voice from other sounds, e.g. http://www.izotope.com/products/audio/rx/. If you're recording voice in stereo, you may be successful by analyzing the left/right position of the voice, and then removing non-voice components by correlating left and right.
Re: mkb's comments:
- a noise gate will mute audio when the level (of some or all frequency components) is below a threshold. Depending on the level of your background noise, this might help, but often a noise gate is annoying because you'll hear the background as long as your "good" audio is there, and there's silence in between.
- a low pass filter removes high frequency components (it lets low frequencies pass)! So if your background noise is of high frequency, it might help. A simple FIR or IIR low pass filter is easy to implement. But it will also affect your "good" audio, if it has high frequency components.