Sending a password over SSH or SCP with subprocess.Popen

Question

I'm trying to run an scp (secure copy) command using subprocess.Popen. The login requires that I send a password:

from subprocess import Popen, PIPE

proc = Popen(['scp', "[email protected]:/foo/bar/somefile.txt", "."], stdin = PIPE)
proc.stdin.write(b'mypassword')
proc.stdin.flush()

This immediately returns an error:

[email protected]'s password:
Permission denied, please try again.

I'm certain the password is correct. I easily verify it by manually invoking scp on the shell. So why doesn't this work?

Note, there are many similar questions to this, asking about subprocess.Popen and sending a password for automated SSH or FTP login:

How can I set a users password in linux from a python script?
Use subprocess to send a password

The answer(s) to these questions don't work and/or don't apply because I am using Python 3.

Douche

i'm not familiar with this code but you try to use -p parameter? also a key-exchange as a validation method? – Epsilon

With -p i mean "-p mypassword [email protected]:/foo/bar/somefile.txt" i don't know if you can do that – Epsilon

@Epsilon As far as I know, scp does not accept a switch to input the password over the command line(its man page contains nothing of the sort). In general it is not good practice to supply a password on the command line because that will get recorded in .bash_history – Papua

Perhaps you can solve this problem by creating an authorized_keys file. – Sweetandsour

Doesn't scp need a -r if you're going to be copying . (a directory)? If you're struggling to get pexpect to work, I've added a function below that should help. – Stockman

Answer 1

The second answer you linked suggests you use Pexpect (which is usually the right way to go about interacting with command line programs that expect input).

Papua

note: regular pexpect supports Python 3 now. – Amar

Answer 2

Here's a function to ssh with a password using pexpect:

import pexpect
import tempfile

def ssh(host, cmd, user, password, timeout=30, bg_run=False):                                                                                                 
    """SSH'es to a host using the supplied credentials and executes a command.                                                                                                 
    Throws an exception if the command doesn't return 0.                                                                                                                       
    bgrun: run command in the background"""                                                                                                                                    
                                                                                                                                                                               
    fname = tempfile.mktemp()                                                                                                                                                  
    fout = open(fname, 'w')                                                                                                                                                    
                                                                                                                                                                               
    options = '-q -oStrictHostKeyChecking=no -oUserKnownHostsFile=/dev/null -oPubkeyAuthentication=no'                                                                         
    if bg_run:                                                                                                                                                         
        options += ' -f'                                                                                                                                                       
    ssh_cmd = 'ssh %s@%s %s "%s"' % (user, host, options, cmd)                                                                                                                 
    child = pexpect.spawn(ssh_cmd, timeout=timeout)  #spawnu for Python 3                                                                                                                          
    child.expect(['[pP]assword: '])                                                                                                                                                                                                                                                                                               
    child.sendline(password)                                                                                                                                                   
    child.logfile = fout                                                                                                                                                       
    child.expect(pexpect.EOF)                                                                                                                                                  
    child.close()                                                                                                                                                              
    fout.close()                                                                                                                                                               
                                                                                                                                                                                                                                                                                                                        
    fin = open(fname, 'r')                                                                                                                                                     
    stdout = fin.read()                                                                                                                                                        
    fin.close()                                                                                                                                                                
                                                                                                                                                                               
    if 0 != child.exitstatus:                                                                                                                                                  
        raise Exception(stdout)                                                                                                                                                
                                                                                                                                                                               
    return stdout

Something similar should be possible using scp.

Stockman

Thanks. This helped me out. I'm looking it up, but could you tell me what this line means? '-q -oStrictHostKeyChecking=no -oUserKnownHostsFile=/dev/null -oPubkeyAuthentication=no' – Vat

@Vat to these are options you would pass to ssh on the command line or in your .ssh/config without the -o. The first tells ssh not to complain if the host has changed its key cos it may have been reinstalled since you last used this. The second says to use /dev/null rather than .ssh/known_hosts so again it will stop it complaining if it's been reinstalled with a duplicate entry warning. The final days not to use keys which forces password entry. – Stockman

For python 3, spawnu should be used instead of spawn according to https://mcmap.net/q/822260/-typeerror-when-calling-expect-method-of-pexpect-module-in-python-3 to avoid write() argument must be str, not bytes – Mispleading

import tempfile and import pexpect should be added, pip install pexpect or pip3 install pexpect can install the required package – Mispleading

Answer 3

The OpenSSH scp utility invokes the ssh program to make the SSH connection to the remote host, and the ssh process handles authentication. The ssh utility doesn't accept a password on the command line or on its standard input. I believe this is a deliberate decision on the part of the OpenSSH developers, because they feel that people should be using more secure mechanisms like key-based authentication. Any solution for invoking ssh is going to follow one of these approaches:

1. Use an SSH key for authentication, instead of a password.
2. Use sshpass, expect, or a similar tool to automate responding to the password prompt.
3. Use (abuse) the SSH_ASKPASS feature to get ssh to get the password by invoking another command, described here or here, or in some of the answers here.
4. Get the SSH server administrator to enable host-based authentication and use that. Note that host-based authentication is only suitable for certain network environments. See additional notes here and here.
5. Write your own ssh client using perl, python, java, or your favorite language. There are ssh client libraries available for most modern programming languages, and you'd have full control over how the client gets the password.
6. Download the ssh source code and build a modified version of ssh that works the way you want.
7. Use a different ssh client. There are other ssh clients available, both free and commercial. One of them might suit your needs better than the OpenSSH client.

In this particular case, given that you're already invoking scp from a python script, it seems that one of these would be the most reasonable approach:

1. Use pexpect, the python expect module, to invoke scp and feed the password to it.
2. Use paramiko, the python ssh implementation, to do this ssh task instead of invoking an outside program.

Flume

Answer 4

Pexpect has a library for exactly this: pxssh

http://pexpect.readthedocs.org/en/stable/api/pxssh.html

import pxssh
import getpass
try:
    s = pxssh.pxssh()
    hostname = raw_input('hostname: ')
    username = raw_input('username: ')
    password = getpass.getpass('password: ')
    s.login(hostname, username, password)
    s.sendline('uptime')   # run a command
    s.prompt()             # match the prompt
    print(s.before)        # print everything before the prompt. 
    s.logout()
except pxssh.ExceptionPxssh as e:
    print("pxssh failed on login.")
    print(e)

Hedden

import pxssh \snap install microk8s – Geographical

This solution doesn't seem to work on windows: #36874440 – Uvarovite

Answer 5

I guess some applications interact with the user using stdin and some applications interact using terminal. In this case when we write the password using PIPE we are writing to stdin. But SCP application reads the password from terminal. As subprocess cannot interact with user using terminal but can only interact using stdin we cannot use the subprocess module and we must use pexpect for copying the file using scp.

Feel free for corrections.

Convertiplane

Answer 6

Here is my scp function based on pexpect. It can handle wildcards (i.e. multiple file transfer), in addition to the password. To handle multiple file transfer (i.e. wildcards), we need to issue a command via a shell. Refer to pexpect FAQ.

import pexpect

def scp(src,user2,host2,tgt,pwd,opts='',timeout=30):
    ''' Performs the scp command. Transfers file(s) from local host to remote host '''
    cmd = f'''/bin/bash -c "scp {opts} {src} {user2}@{host2}:{tgt}"'''
    print("Executing the following cmd:",cmd,sep='\n')

    tmpFl = '/tmp/scp.log'
    fp = open(tmpFl,'wb')
    childP = pexpect.spawn(cmd,timeout=timeout)
    try:
        childP.sendline(cmd)
        childP.expect([f"{user2}@{host2}'s password:"])
        childP.sendline(pwd)
        childP.logfile = fp
        childP.expect(pexpect.EOF)
        childP.close()
        fp.close()

        fp = open(tmpFl,'r')
        stdout = fp.read()
        fp.close()

        if childP.exitstatus != 0:
            raise Exception(stdout)
    except KeyboardInterrupt:
        childP.close()
        fp.close()
        return

    print(stdout)

It can be used this way:

params = {
    'src': '/home/src/*.txt',
    'user2': 'userName',
    'host2': '192.168.1.300',
    'tgt': '/home/userName/',
    'pwd': myPwd(),
    'opts': '',
}

scp(**params)

Ecstatics

Answer 7

This is a rewrite I did from the code posted by @Kobayashi and @sjbx but for the purposes of doing scp requests, so credit to those two.

def scp(host, user, password, from_dir, to_dir, timeout=300, recursive=False):
    fname = tempfile.mktemp()
    fout = open(fname, 'w')

    scp_cmd = 'scp'
    if recursive:
        scp_cmd += ' -r'
    scp_cmd += f' {user}@{host}:{from_dir} {to_dir}'
    child = pexpect.spawnu(scp_cmd, timeout=timeout)
    child.expect(['[pP]assword: '])
    child.sendline(str(password))
    child.logfile = fout
    child.expect(pexpect.EOF)
    child.close()
    fout.close()

    fin = open(fname, 'r')
    stdout = fin.read()
    fin.close()

    if 0 != child.exitstatus:
        raise Exception(stdout)

    return stdout

Nonna