The main use of asp-append-version=true is to bypass the browser cache if the content item has changed. It is very useful for javascript that is likely to change.
Images don't get updated very often unless you have an unusual use case. If you have a use case where images do get updated and they keep the same file name then it would make sense to use it there but otherwise it would not.
I can't say how much performance hit would be involved, but it does create a hash of the file content to use for the version string that is appended. There is some impact for any extra code that is executed when it is not needed. It does not make sense for it to be a default behavior, it should be used by opting in to it if you really need it.