I think I have the beginnings of an answer here. First of all, you need to be able to build Emacs on some machine with the same architecture as the runtime machine. You could get around this with cross compiling, but that makes everything way more complicated.
tar xf emacs.tar.bz2
cd emacs
./autogen.sh
./configure --with-x=no --prefix=/usr/local
make
mkdir install
make DESTDIR=$PWD/install install
You can make prefix whatever you want, it just has to be where you're going to have Emacs installed on the runtime machine. X is disabled because it drastically reduces the number of libraries required and you're running over ssh anyway.
Now figure out what shared libs are needed to run Emacs. A quick look at all of the executables shipped with Emacs shows that this is a superset of the libs needed for the entire Emacs installation.
$ ldd install/usr/local/bin/emacs
linux-vdso.so.1 (0x00007fff427fe000)
libasound.so.2 => /usr/lib/libasound.so.2 (0x00007f66b25a0000)
librt.so.1 => /usr/lib/librt.so.1 (0x00007f66b2398000)
libdbus-1.so.3 => /usr/lib/libdbus-1.so.3 (0x00007f66b2151000)
libxml2.so.2 => /usr/lib/libxml2.so.2 (0x00007f66b1de9000)
libgpm.so.2 => /usr/lib/libgpm.so.2 (0x00007f66b1be2000)
libncursesw.so.5 => /usr/lib/libncursesw.so.5 (0x00007f66b1983000)
libgnutls.so.28 => /usr/lib/libgnutls.so.28 (0x00007f66b1673000)
libpthread.so.0 => /usr/lib/libpthread.so.0 (0x00007f66b1457000)
libm.so.6 => /usr/lib/libm.so.6 (0x00007f66b1159000)
libc.so.6 => /usr/lib/libc.so.6 (0x00007f66b0dac000)
libdl.so.2 => /usr/lib/libdl.so.2 (0x00007f66b0ba8000)
/lib64/ld-linux-x86-64.so.2 (0x00007f66b2897000)
libz.so.1 => /usr/lib/libz.so.1 (0x00007f66b0992000)
liblzma.so.5 => /usr/lib/liblzma.so.5 (0x00007f66b076f000)
libp11-kit.so.0 => /usr/lib/libp11-kit.so.0 (0x00007f66b054e000)
libtasn1.so.6 => /usr/lib/libtasn1.so.6 (0x00007f66b033a000)
libnettle.so.4 => /usr/lib/libnettle.so.4 (0x00007f66b010c000)
libhogweed.so.2 => /usr/lib/libhogweed.so.2 (0x00007f66afedd000)
libgmp.so.10 => /usr/lib/libgmp.so.10 (0x00007f66afc66000)
So copy all of these libs to some directory in your install tree. linux-vdso.so
is virtual and can't/doesn't need to be copied.
mkdir install/usr/local/solib
cp /usr/lib/libasound.so.2 /usr/lib/librt.so.1 /usr/lib/libdbus-1.so.3 /usr/lib/libxml2.so.2 /usr/lib/libgpm.so.2 /usr/lib/libncursesw.so.5 /usr/lib/libgnutls.so.28 /usr/lib/libpthread.so.0 /usr/lib/libm.so.6 /usr/lib/libc.so.6 /usr/lib/libdl.so.2 /lib64/ld-linux-x86-64.so.2 /usr/lib/libz.so.1 /usr/lib/liblzma.so.5 /usr/lib/libp11-kit.so.0 /usr/lib/libtasn1.so.6 /usr/lib/libnettle.so.4 /usr/lib/libhogweed.so.2 /usr/lib/libgmp.so.10 install/usr/local/solib/
Archive it all. I've been liking squashfs lately, use tar if you prefer.
mksquashfs install/usr/local emacs.sfs -noappend
On the runtime machine, extract your files and copy them to the prefix. With squashfs, we can just mount it.
mount emacs.sfs /usr/local
Start emacs with the LD_LIBRARY_PATH
set to use the libraries you copied earlier.
LD_LIBRARY_PATH=/usr/local/solib /usr/local/bin/emacs
Hopefully that will work. I tested in a VM with a fairly similar OS to the one I built on, so maybe something will go awry when they differ by a lot.
C-x C-f /ssh:...
Though, even if that works for you, I'd still love to see someone post instructions for a 'portable' emacs. – Inestimable