Configure MPI hostsfile to use multiple user identities

Question

I want to run a program with mpirun on different sets of machines (All linux machines with Open Mpi 1.5).

Right now I have machines where I can log on with username A, and another set of machines where I use the username B. All machines are accessible via ssh, but I can't figure out how to achieve this.

My hosts file would be like this :

localhost          #username local

machine_set_A_1    #username A
machine_set_A_2    #username A
...

machine_set_B_1    #username B
machine_set_B_2    #username B
...

Is it possible to achieve this. Thank you.

Av

All linux machines with Open Mpi 1.5, question updated – Av

Answer 1

The OpenSSH client supports per-host configurations, something similar to this:

Host machine_set_A_1 machine_set_A_2 ...
User username_A

Host machine_set_B_1 machine_set_B_2 ...
User username_B

The Host directive restricts all the following declarations (up to the next Host directive) to apply only to connections made to hostnames, that match any of the patterns given after the directive.

The SSH client configuration file is usually found in /etc/ssh/ssh_config (system-wide configuration) and in ~/.ssh/config (user-specific configuration; directives there override those from the system configuration).

The man page for ssh_config(5) has more information on the available keywords and more information about what patterns are supported (using something like Host *.groupA.uni.edu, if applicable, could save some typing).

Shoer

@JonathanDursi, your answer was applicable to the more general case as the wrapper could also do other different things than to simply specify the correct username (e.g. provide integration with something like SGE). You shouldn't have it deleted. – Shoer

it seems hack-y, in retrospect, but if you think it might be useful in some other cases I'll un-delete. – Avian

Answer 2

The only way I can think of to do this is to create a wrapper for your process launcher to do ssh user1@hostname for some, and user2@hostname for others. You can set the process launcher ssh agent via

mpirun -mca orte_rsh_agent "/path/to/mysshwrapper" -machinefile machines.txt -np 4 ./subarray

and then you'd have to inject the appropriate username arguments into the call to ssh from your wrapper.

You'll also of course have to have ssh keys setup so that your login to the other username accounts will be passwordless.

Avian

One can create an SSH client configuration file and specify there different host configurations with the Host keyword. For each host one can specify the login username with with the User keyword. Should be somewhat simpler than creating a wrapper script. – Shoer

@HristoIliev Yeah, that's a better answer both in simplicity and usefulness (it's presumably helpful even outside of mpirun). You ought to post that as a proper answer rather than a comment on this one so it gets the visibility it deserves. – Avian

Answer 3

Trying to find solution for my own problem I have just found something like that (on http://wiki.mpich.org/mpich/index.php/Using_the_Hydra_Process_Manager). It might be helpful for you, but this is for MPICH.

Using Hydra on Machines with Different User Names

Hydra only supports using different usernames on some launchers (such as ssh and rsh). For this, the host file should contain a "user=" entry.

An example

shell$ cat hosts

       donner  user=foo
       foo     user=bar
       shakey  user=bar

EDIT

For OpenMPI please refer to: http://www.open-mpi.org/faq/?category=rsh#rhosts-file

Mook

Answer 4

Normally you'd just install mpi on each of the machines you want to run mpi jobs on. Then in your mipexec call you'd specify the machinefile that identifies the hosts/machines that have mpi. The mpi smpd/daemon will start the jobs for you on each host.

Example:

 mpiexec -f machinefile -n 32 a.out

Machine file looks like:

 host1.some.place:16
 host2.some.place:16

Which means two hosts with 16 cores each.

refer to:

mpich2 documentation

Caret

and how it will know what user to use in order to execute commands on the machines? my problem is that I have different users! – Av