Azure DevOps - Clean build directory

Question

I am using a self-hosted agent for running my build and release pipelines.

A problem happens after I run a build and it fails due to an issue in the pipeline. The cloned branch from remote sits in the working directory. The second run also uses the same working directory, not a new one. I have verified on the agent that no new directory was created. I can also see in the Azure Pipeline logs that it's using the same old directory.

What I did temporarily to solve this was I located the old working directory (e.g. /home/user/_work/13) and manually deleted it. Then Azure creates a new folder (e.g. "14") under _work, and I could see the latest cloned code from the remote in that new directory.

How can I automatically delete the working folder (the numbered directory under _work) when a failure occurs?

Note: I already have put clean-up steps in place at the end of my pipeline which run when the pipeline completes with success. Also, I am writing YAML pipelines, not Classic.

Please let me know if any information is required for a better understanding.

Cumbrance

Are you using Git or TFVC? Are you using YAML pipelines or classic pipelines? Did you look at the documentation for working folder cleaning options? – Poop

I am using the azure git repository, I am not getting why someone down voted this, if they found it silly why do not they answer it .. – Cumbrance

@DanielMann, let me know if anything else you required for better understanding. thank you – Cumbrance

Answer 1

Azure DevOps - Clean build directory

There is a Clean option on the Get Sources tab, which can perform different kinds of cleaning of the working directory of your private agent before the build is run:

We could set the value to true to clean the working directory of your private agent. Even if the build is failed.

You could check the document Clean the local repo on the agent for some more details.

Update:

But this is meant for the classic pipeline, do not we have any tag which we define in yml pipeline only

jobs:
- job: string  # name of the job (A-Z, a-z, 0-9, and underscore)
  ...
  workspace:
    clean: outputs | resources | all # what to clean up before the job runs

Check this document YAML schema reference for some details.

Antecedent

Thank you for a response, but this is for the classic pipeline, do not we have any tag which we define in yml pipeline only – Cumbrance

@YouAreAwesome, hm, you didn’t mention a bit about YAML in your original question, so I thought you were in classic mode. Now, I have updated the answer for your comment, please check if it help you. – Antecedent

I missed from the document somehow, thank you for the reference link appreciated. – Cumbrance

I could not find any information about cleaning up the workspace directory in the YAML schema reference link provided in your answer. However you did point me in the right direction and I found on google the following Azdo page describing how to do it: learn.microsoft.com/en-us/azure/devops/pipelines/process/… – Doctrinaire

How do we clean after the job? – Facsimile

This option is not present on release pipelines... – Magdalenmagdalena

@IndrajeetGour the option in the screenshot is available for yaml pipelines... – Ula

Answer 2

Since running pre-cleanup-jobs (workspace.clean) or post-cleanup-jobs some step X to cleanup the agent workdir heavily relies on the pipeline-author of "some pipeline" to fix a potential hard to detect "workdir pollution" issue, we decided to re-implement the idea to guarantee workdir pollution can never happen.

I have created https://github.com/EugenMayer/azure-agent-self-hosted-toolkit which fixes this project on the agent-level, not requiring any changes to the pipelines nor relying on those.

To quote to project idea

The run-once mode is based on Microsoft's `./run.sh --once` which ensures that an agents only runs 1 job and then stops.
This is used to

 - cleanup the workdir in a safe manner after each job
 - ensures each job on an agent runs in a clean workdir
 - starts the agent right after cleanup up (few seconds) to be available for the next job

This repository also offers a toolkit to start / setup x-agents and maintain them using the original tools of Microsoft, but wrapped in convenient scripts.

If this helps anybody else, happy to share it.

Autarky

Answer 3

The simplest way of doing this was to add a step at the end of the pipeline with the following configuration:

  - task: DeleteFiles@1
    condition: always()
    inputs:
      SourceFolder: "$(Agent.BuildDirectory)"
      RemoveDotFiles: true
      RemoveSourceFolder: false
      Contents: |
        a
        b
        s/**/*
        TestResults

The step excludes the deletion of the s directory to avoid the error related to a process using it. But it removes the entire content

Quicken

Thank you, this was exactly what I needed. – Phalanger