Install Python to Self-hosted Windows build agent

Question

I have installed Windows agent and I need to be able run Python scripts. I know I need to install Python, but I have no idea how.

I added Python files from standard installation to

$AGENT_TOOLSDIRECTORY/
    Python/
        3.8.2/
            x64/
                {tool files}
            x64.complete

Restarted agent, but what now? How to put it to Capabilities? What I'm missing?

EDIT: I need to run this YAML task

steps:
- task: UsePythonVersion@0
  inputs:
    versionSpec: '3.x'
    addToPath: true

- script: |
    python -m pip install --upgrade pip
    pip install -r requirements.txt
  displayName: 'Install dependencies'

- task: BatchScript@1
  displayName: 'Run script make.bat'
  inputs:
    filename: make.bat
    arguments: html

Amberambergris

Answer 1

I have set up a self-hosted agent on a Windows 10 laptop, (for which I have admin access), and I'm running Azure DevOps Express 2020.

I found, downloaded and installed an agent according to the instructions at Download and configure the agent. I used vsts-agent-win-x64-2.170.1.zip and set this up to run as a service, (I guess anyone running it manually needs to double check that it's runnning at show time). I also ran the install command as admin in powershell!

To install a Python version I need to download the appropriate installer from the ftp site at Python.org, eg. for 3.7.9 I've used python-3.7.9-amd64.exe. I then run this from the command line (CMD run as administrator) without UI with: python-3.7.9-amd64.exe /quiet InstallAllUsers=0 TargetDir=$AGENT_TOOLSDIRECTORY\Python\3.7.9\x64 Include_launcher=0 (other options for install available in python docs)

Once this is complete, (and it runs in background so will take longer than the initial command), you need to create an empty {platform}.complete file (as described here), in my case this is x64.complete.

This then worked! I did restart the server for this first version, but I've added other python versions since without needing to. My pipeline task was simply:

steps:
- task: UsePythonVersion@0
  displayName: 'Use Python $(python.version)'
  inputs:
    versionSpec: '$(python.version)'

(with a variable python.version set us as a list of versions 3.7.9, 3.8.8)

One key element for me was the file structure, where the documentation says {tool files} this means the python.exe file and other common dirs such as Lib and Scripts. I initially installed these in a sub-dir which didn't work. So it should look like this:

$AGENT_TOOLSDIRECTORY/
    Python/
        3.7.9/
            x64/
                Doc/
                Lib/
                Scripts/
                python.exe
                ...etc...
            x64.complete

To be honest I'm mostly relieved that this worked without too much trouble. I gave up trying to get Artifacts to work on-prem. In my limited experience all of this is much easier, and better, on the cloud version. Haven't yet persuaded my employer to take that leap however...

Pence

The version folder has be named in the format x.x.x e.g. 3.9.0 and not 3.9 – Seise

Answer 2

The way I install python in my own self-hosted agent that you can refer.

parameters:
- name: projecWorkingDirectory
  displayName: 'Working directory of project'
  default: $(System.DefaultWorkingDirectory)

- name: pythonVersion
  displayName: 'Python version'
  default: '3.10.12'

steps:
- script: |
    mkdir -p $(Agent.ToolsDirectory)/Python/${{ parameters.pythonVersion }}/x64
    mkdir -p $(Agent.ToolsDirectory)/Python/${{ parameters.pythonVersion }}/x64.complete
  displayName: 'Create Python local folders'

- script: |
    curl -LO https://github.com/actions/python-versions/releases/download/3.10.12-5200619051/python-3.10.12-linux-22.04-x64.tar.gz
    tar -xvf python-3.10.12-linux-22.04-x64.tar.gz
  workingDirectory: $(Agent.ToolsDirectory)/Python/${{ parameters.pythonVersion }}/x64
  displayName: 'Downloaded'

- task: UsePythonVersion@0
  inputs:
    versionSpec: ${{ parameters.pythonVersion }}
    architecture: 'x64'
    disableDownloadFromRegistry: true
  displayName: 'Use Python ${{ parameters.pythonVersion }}'

- script: |
    python -m pip install --upgrade pip
    pip install -r requirements.txt
  workingDirectory: ${{ parameters.projecWorkingDirectory }}
  displayName: 'Install dependencies'

Subclimax

Thank you for contributing to the Stack Overflow community. This may be a correct answer, but it’d be really useful to provide additional explanation of your code so developers can understand your reasoning. This is especially useful for new developers who aren’t as familiar with the syntax or struggling to understand the concepts. Would you kindly edit your answer to include additional details for the benefit of the community? – Streaky

The last script in your given azure pipeline fails for me. I am running in a self-hosted agent in a linux VM. It seems that it cannot find python. – Horselaugh

Answer 3

For this issue, in order to use the python version installed in your on-premise machine. You either need to point to the python.exe physical path in cmd task. Or add the python.exe path to environment variable path manually in powershell task. For example:

To use local python in powershell task:

$env:Path += ";c:\{local path to}\Python\Python38\; c:\{local path to}\Python\Python38\Scripts\"
python -V

To use python in CMD task:

c:\{local path to}\Python\Python38\python.exe -V
c:\{local path to}\Python\Python38\Scripts\pip.exe install

So, I think to run python script with private agent, just make sure python is installed locally, then point to python.exe path. You can refer to this case for details.

Stafani

Well, I don't understand. Everywhere is written that I have to put Python to tools directory. So do I need installl it and put to Tools? Or just install and ignore Tools? I need to run simple YAML task (added to OP). – Amberambergris

Answer 4

I added these 4 Tasks before being able to execute python on my pipeline with a vs2017-win2016 agent:

Use Python 3.x

steps:
- task: UsePythonVersion@0
  displayName: 'Use Python 3.x'

Use Pip Authenticate

steps:
- task: PipAuthenticate@1
  displayName: 'Pip Authenticate'

Use Commandline task

steps:
- script: |
   python -m pip install --upgrade pip setuptools wheel


  failOnStderr: true
  displayName: 'install pip for setup of python framework'

Use Commandline task

steps:
- script: 'pip install -r _python-test-harness/requirements.txt'
  failOnStderr: true
  displayName: 'install python framework project''s specific requirements'

Hope that helps

Finfoot

the question is about a self-hosted agent. The azure agent vs2017-win2016 will already have the Python capability I believe. – Conlon