A pure Swift application which is not importing any framework can now be compiled for iOS, OS X and for Linux.
You will generate different executables, because it's different platforms, but the code source can be the same, it just has to be compiled for the respective platform.
The difference is when you import frameworks.
If you import UIKit to make an iOS application, obviously you won't be able to compile this on Linux, because Linux doesn't have those iOS UIKit libraries, they're only available on a Mac via Xcode.
If you import Cocoa to make an OS X application, the same logic applies, it works only for OS X.
The same way, you can make Swift code that only works on Linux if you import specific Linux libraries, like Glibc, that won't work on the Mac.
Etc, etc.
What you need to remember is that the programming language isn't that relevant when it comes to make applications for a platform, it's the frameworks and libraries that you're linked to that are decisive.
Different Swift versions come with different compilers (different toolchains, actually) so to answer directly, no you can't compile for Linux Swift with the normal OS X Swift compiler, you have to use the Swift.org's one.