Azure DevOps: Attach files to test result / run from YAML pipeline

Question

I have some (Playwright) automated tests that run in Azure DevOps via a YAML pipeline.

I have enabled the trace feature of Playwright so each test produces a zip file containing the trace. My goal is for failing tests to have the trace zip file attached to the test result, allowing us to easily click through and see exactly what went wrong. These files can be up to about 15MB.

This is where I want them to appear in Azure DevOps:

Looking at the documentation I see there is a REST API to post attachments, but I'm unclear how to invoke this API from the YAML pipeline. i.e. authenticating for the API, getting the test case ID & sending the zip file as base64.

I guess I'm looking for an example YAML task that does this.

Thoroughwort

Answer 1

Update:

Hi, some update on the previous reply, by intercepting network traffic, I found an easier way than REST API. First of all, please refer to this document, the buildid of the current pipeline can be easily obtained through the predefined variable Build.BuildId. Then, please run this request by get method to get test results(Based on your build pipeline run): https://vstmr.dev.azure.com/<orgname>/<projectname>/_apis/testresults/resultsbypipeline?pipelineId=<buildid>

There is no official documentation or reference for this method, but it works and I have tested it.

Original Answer:

but I'm unclear how to invoke this API from the YAML pipeline. i.e. authenticating for the API, getting the test case ID & sending the zip file as base64.

You can use any language/script language you want to send the request.

For the authentication, you can use the PAT token, here are detailed steps:

https://learn.microsoft.com/en-us/azure/devops/organizations/accounts/use-personal-access-tokens-to-authenticate?view=azure-devops&tabs=Windows

For example, if you use Python,

This is how to write zip file via base64(Python):

base64 encode a zip file in Python

And then use below code to upload the zip file.

import http.client
import json

conn = http.client.HTTPSConnection("dev.azure.com")
payload = json.dumps({
  "Stream": <zip stream data here>,
  "FileName": "test.zip"
})
headers = {
  'Authorization': 'Basic <Your PAT Token here>',
  'Content-Type': 'application/json'
}
conn.request("POST", "/<orgname>/<projectname>/_apis/test/Runs/<runid>/Results/<testresultid>/attachments?api-version=6.0-preview.1", payload, headers)
res = conn.getresponse()
data = res.read()
print(data.decode("utf-8"))

For test case id, you can get it in the workitem url:

Or you can use this REST API to list the test cases in test suits:

https://learn.microsoft.com/en-us/rest/api/azure/devops/test/test-suites/get-test-cases?view=azure-devops-rest-5.0

YAML pipeline should be like this:

pool:
  name: Azure Pipelines
steps:
- task: PythonScript@0
  displayName: 'Run a Python script'
  inputs:
    scriptPath: yourpythonfilename.py

Just a simple, for how to do it on your side, you need to design it by yourself.

Selfinsurance

This is great - really comprehensive & will get me on my way. One extra question - I said test case in error, what I meant was test result. There will be a zip file per test, and the zip file will have the same name as the test method. Is there a way of querying test results by name then uploading the corresponding zip to the test result? – Thoroughwort

There doesn't have built-in method to achieve. But these two API will help you: Get Test Runs List Results – Selfinsurance

Thanks, yes I saw those APIs - but I guess the question is how do I get the current test run ID from within the pipeline – Thoroughwort

@DavidMasters I found an easier way, StackOverflow said my comment is too long, so I update the answer. – Selfinsurance

Thank you so much for taking the time on this! – Thoroughwort