Running multiple exec tasks in parallel

Question

I am batching multiple exec tasks in the build process. Each execution takes around one minute to complete, so I would like to run them in parallel to improve overall build performance.

The target that run multiple exec tasks:

<Target Name="CreatePackages" DependsOnTargets="Build" AfterTargets="Build">
  <Exec Command="SomeExecutable.exe %(SomeGroup.Location) </Exec>
</Target>

The ItemGroup definition:

<ItemGroup>
  <SomeGroup Include="Production">
    <Location>SomePath/Production</Location>
  </SomeGroup>
  <SomeGroup Include="Test">
    <Location>SomePath/Test</Location>
  </SomeGroup>
  <SomeGroup Include="Development">
    <Location>SomePath/Development</Location>
  </SomeGroup>
</ItemGroup>

How do I run these exec tasks in parallel?

Deflection

Answer 1

MSBuild doesn't parallelize on task or target level, but on project level only. You choices are to wrap each individual location in a target and call project itself in parallel (messy), or to write a custom task with TPL (System.Threading.Tasks.Parallel) and Microsoft.Build.Tasks.Exec or System.Diagnostics.Process (easy), or try YieldDuringToolExecution (technically not parallelization, but other tasks would wait less).

Isaac

You can run targets in parallel.. Unless I am somehow very confused! – Deflection

@davenewza Sort of, look at the screenshoot, it has 3 build summaries. I think it'll be more accurate to say that it built the project thrice in parallel with one target each rather than it built project with three parallel targets, and that is exactly what I meant under the first option, although I'd recommend you do the second one with TPL and custom task. – Isaac

@davenewza, technically MSBuild only parallelizes on project level only. The trick with the Extension Pack is that it launches several instances of MSBuild.exe in parallel, equivalent to calling msbuild MyProject.proj /t:OneOfMyTargets. This might look similar to MSBuild natively supported parallelization, but it is not. The reason is any pre-requisites of your targets will get executed multiple times, thus breaking MSBuild model. MSBuild (on project level) guarantees any target gets executed at most once. I would not recommend using that functionality from extension pack. – Knave