Build step triggered by TeamCity always builds - even when there are no changes

Question

The problem: I am setting up TeamCity as a build server for an ASP.NET MVC project. I am using Powershell with psake to run msbuild against our .csproj file and create a deployable package. From the build server, I can open up powershell, run the script and, because there are no source code changes, msbuild does not regenerate the project DLL files. BUT, when I call the exact same script from the TeamCity web interface, msbuild ALWAYS rebuilds and regenerates the DLL files even though there are no changes. Not what it should do AFAIK.

I have narrowed this problem down to a single step. To keep it simple, I have set up my TeamCity config so it is not using any source control, it runs a single "powershell" build step that calls my powershell script.

The powershell script runs a single command:

exec { &$msbuild $ProjectFile /t:Package "/p:PackageLocation=$PackageFile;OutDir=$TempPath;Configuration=$Config;SolutionDir=$BaseDir\Source\" /v:m }

When I call the script manually from a powershell command line, I see:

CoreCompile:
Skipping target "CoreCompile" because all output files are up-to-date with respect to the input files.

When I call the exact same script through TeamCity, I see:

[11:11:26]: CoreCompile:
[11:11:26]:   c:\WINDOWS\Microsoft.NET\Framework\v4.0.30319\Csc.exe /noconfig ...
<SNIP>
[11:11:32]: CopyFilesToOutputDirectory:
[11:11:32]:   Copying file from "obj\Demo\Website.Web.dll" to "d:\deploy\Build\package\Demo\temp\Website.Web.dll".
[11:11:32]:   Website.Web -> d:\deploy\Build\package\Demo\temp\Website.Web.dll
[11:11:32]:   Copying file from "obj\Demo\Website.Web.pdb" to "d:\deploy\Build\package\Demo\temp\Website.Web.pdb".
[11:11:32]: _CopyWebApplicationLegacy:
[11:11:32]:   Copying Web Application Project Files for Website.Web
[11:11:32]:   Copying file from "obj\Demo\Website.Web.dll" to "d:\deploy\Build\package\Demo\temp\_PublishedWebsites\Website.Web\bin\Website.Web.dll".
[11:11:32]:   Copying file from "obj\Demo\Website.Web.pdb" to "d:\deploy\Build\package\Demo\temp\_PublishedWebsites\Website.Web\bin\Website.Web.pdb".
[11:11:32]:   Copying file from "d:\deploy\Build\package\Demo\temp\Website.Data.dll" to "d:\deploy\Build\package\Demo\temp\_PublishedWebsites\Website.Web\bin\Website.Data.dll".
[11:11:32]:   Copying file from "d:\deploy\Build\package\Demo\temp\Website.Data.pdb" to "d:\deploy\Build\package\Demo\temp\_PublishedWebsites\Website.Web\bin\Website.Data.pdb".

Any ideas why running this script from TeamCity causes msbuild to detect changes and rebuild, but running the exact same script manually does not?

UPDATE: Thinking this might be caused by some quirk with the TeamCity Powershell runner, I just tried making a batch file that passes the script into Powershell.exe and called it using the Command Line runner:

C:\WINDOWS\system32\WindowsPowerShell\v1.0\powershell.exe -NonInteractive -File D:\deploy\Build\run-build.ps1 && exit /b %ERRORLEVEL%

and I get the exact same behavior. If I call this batch file from the command line, the msbuild skips compilation. If I call it from TeamCity, the DLLs are recompiled.

UPDATE #2: Eureka! I turned on diagnostic debugging in msbuild and found the cause of the forced recompile. It is caused by the GenerateTargetFrameworkMonikerAttribute target. Here is the key bits from the log output:

[15:23:28]: Target "GenerateTargetFrameworkMonikerAttribute" in file "c:\WINDOWS\Microsoft.NET\Framework\v4.0.30319\Microsoft.Common.targets" from project "d:\deploy\source\Website.Data\Website.Data.csproj" (target "BeforeCompile" depends on it):
[15:23:28]: Building target "GenerateTargetFrameworkMonikerAttribute" completely.
[15:23:28]: Output file "C:\TeamCity\buildAgent\temp\buildTmp\.NETFramework,Version=v4.0.AssemblyAttributes.cs" does not exist.
[15:23:28]: Using "WriteLinesToFile" task from assembly "Microsoft.Build.Tasks.v4.0, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a".
[15:23:28]: Task "WriteLinesToFile"
[15:23:28]: Done executing task "WriteLinesToFile".
[15:23:28]: Done building target "GenerateTargetFrameworkMonikerAttribute" in project "SMM.Data.csproj".

It looks like this target creates/updates an AssemblyAttributes file in the TEMP dir as specified in the TEMP environment variable. Apparently TeamCity overrides the TEMP environment variable and sets it to: C:\TeamCity\buildAgent\temp\buildTmp and this directory is cleaned before every build.

I can see this if I call Get-ChildItem Env: from powershell:

TEMP                           C:\DOCUME~1\ADMINI~1\LOCALS~1\Temp
TMP                            C:\DOCUME~1\ADMINI~1\LOCALS~1\Temp

But if I call it from the powershell script as called from TeamCity:

TEMP                           C:\TeamCity\buildAgent\temp\buildTmp            
TMP                            C:\TeamCity\buildAgent\temp\buildTmp   

The key piece is that after this file is regnerated:

[15:23:28]: Building target "CoreCompile" completely.
[15:23:28]: Input file "C:\TeamCity\buildAgent\temp\buildTmp\.NETFramework,Version=v4.0.AssemblyAttributes.cs" is newer than output file "obj\Demo\SMM.Data.pdb".

And this is why the whole project is getting recompiled.

When I run the script from Powershell, the temp directory is not changed or cleaned and the build runs as expected.

So, anyone know how I can either change the directory that this AssemblyAttributes file is created, or tell TeamCity to use a different TEMP dir? I have to believe that this is an issue that others have run into.

Thanks!

Fleurdelis

Answer 1

So, as I mentioned in "Update #2" above, the problem seems to be caused by 2 things: - TeamCity sets the TEMP and TMP environment vars to its own temp directory - TeamCity "cleans" this temp directory prior to every build - Part of the msbuild process runs a GenerateTargetFrameworkMonikerAttribute target that updates a specific file in the directory specified by the TEMP environment variable - causing the compiler to thing it needs to recompile the whole project

Once I figured this out, I found an applicable answer in this unrelated question: In Visual Studio 2010 why is the .NETFramework,Version=v4.0.AssemblyAttributes.cpp file created, and can I disable this?

So I added:

<Target Name="GenerateTargetFrameworkMonikerAttribute" />

to both of the projects in my solution that compile to DLLs and it worked.

Fleurdelis

I was able to reproduce the issue, and was coming to similar conclusions. Anyway, good that you found the answer. Accept your answer when you are able to. – Incoherent

Seems TeamCity does what it has to do as depending on temp folders or on any other specific folders is not a good idea in CI. – Retentivity

Answer 2

As a variation of obliojoe's answer, you can backup and restore these files to/from TEMP folder, if you do not want or cannot change the individual project files:

1. First attempt to restore the files from a backup:

   copy temp\*.* %%temp%% /y
   echo AssemblyAttributes restore attempted
   

2. Then perform your build step(s) using TeamCity build runner

3. Backup the files:

   mkdir temp 2> nil
   copy %%temp%%\*AssemblyAttributes.cs temp /y
   echo AssemblyAttributes files saved
   

Both batch files need to run from the same directory.

Do note the final ECHO in these batch files, it is there to guarantee successful exit (error code 0).

Chetchetah