You can put the implications into two buckets.
The first is how it affects your application environment. Using unsafe code requires that your assembly be run in a full trust environment. It's not possible to run in a restricted environment such as certain Click Once security settings. The reason being that unsafe code prevents the CLR from ensuring type safety. Click Once though with no security restrictions should not have a problem.
The second is what it means for the way you code. Using unsafe code typically involves using pointers and in particular, using them to performed advanced marshalling via PInvoke. There is nothing inherently wrong with either of these actions though. It just requires significantly more understanding of the CLR and marshalling than "safe" code does. Object pinning is a great example of knowledge you'd need to have a firm grasp on before you started using these features.
fixed
buffers withunsafe
. Maybe you can edit your question to reflect that. – Dioptase