How to choose between step, job and stages in azure DevOps yaml pipelines?
W

1

34

I converted most of my classic build pipelines to yaml now. The yaml convert tool devop basically treated my classic build pipeline tasks as individual yaml tasks. So now I have one job with around 8 tasks.

When learning yaml, I see some will create multiple jobs and some even create multiple stages. Stages for me seems to be matching for different deployment environments (dev, QA, UAT etc). But I am not sure when should I use multiple jobs?

Say I have two solutions within one repo and would like to build both and my artifact will contain both solution's dlls. For each solution, my tasks will be build, run unit test and package the dlls up.

I can put all these into one job, so it will be sequential tasks to do step by step. Or I can create two jobs, each handle the build of one solution. But is there any benefit of using multiple jobs comparing to one? Seems Microsoft allows your agent to run two jobs in parallel (in theory it will be quicker), but they charge you for that.

Thanks

Wu answered 19/3, 2021 at 9:9 Comment(3)
Have you checked my reply? Is my reply helpful?Alf
I did. However as I don't have self hosted agents yet, not sure I understand you yetWu
If you use Microsoft-hosted agent, you are correct that you need to pay more agents to run two jobs in parallel.Alf
A
47

This hierarchy is reflected in the structure of a YAML file like:

enter image description here

A pipeline is one or more stages that describe a CI/CD process. Stages are the major divisions in a pipeline. The stages "Build this app," "Run these tests," and "Deploy to preproduction" are good examples.

A stage is one or more jobs, which are units of work assignable to the same machine. You can arrange both stages and jobs into dependency graphs. Examples include "Run this stage before that one" and "This job depends on the output of that job."

A job is a linear series of steps. Steps can be tasks, scripts, or references to external templates.

A job is a collection of steps run by an agent or on a server. Jobs can run conditionally and might depend on earlier jobs. Jobs can be of different types, depending on where they run.

  • Agent pool jobs run on an agent in an agent pool. These are the most common type of jobs and they run on an agent in an agent pool. Use demands with self-hosted agents to specify what capabilities an agent must have to run your job.
  • Server jobs run on the Azure DevOps Server. Tasks in a server job are orchestrated by and executed on the server (Azure Pipelines or TFS). A server job does not require an agent or any target computers. Only a few tasks are supported in a server job at present.
  • Container jobs run in a container on an agent in an agent pool. For more information about choosing containers, see Define container jobs.

The situation you mentioned is one usage of Jobs. If you use multiple self-hosted agents, you'll see the benefit, it won't charge and you can use demands to specify what capabilities an agent must have to run your job.

Alf answered 22/3, 2021 at 7:22 Comment(0)

© 2022 - 2025 — McMap. All rights reserved.