The cause of the error is that you have upgraded from MPFR 3.1.x to MPFR 4.0.x, which is ABI-incompatible with MPFR 3.1.x (a MPFR 4.0.x library cannot be used with a program compiled against MPFR 3.1.x). Hence the change from libmpfr.4.dylib (used for MPFR 3.0.x and MPFR 3.1.x) to libmpfr.6.dylib (used for MPFR 4.0.x at least).
What you should have done instead, in order to avoid the error, is to install MPFR 4.0.x without uninstalling libmpfr.4.dylib from MPFR 3.1.x: all new compiled software would use MPFR 4.0.x and libmpfr.6.dylib, but older binaries depending on MPFR would have still been able to run by using libmpfr.4.dylib (a possible exception is when there are two different dependencies on MPFR, via libraries, where both libmpfr.4.dylib and libmpfr.6.dylib could be needed, in which case this might not work).
Now, you should recompile Xcode. If you get Xcode already compiled (from a binary package), make sure that you get either one compiled against MPFR 4.0.x, or one that includes the libmpfr.4.dylib file. If this is not possible, you should reinstall MPFR 3.1.x. Then you should be able to install MPFR 4.0.x without removing libmpfr.4.dylib, as said above.