How do I get msdeploy to create App_Data if it doesn't exist, but not delete any contents of the remote directory?

Question

I have an application setup with the following Package/Publish Web settings:

• Only files needed to run this application
• (unchecked) Exclude generated debug symbols
• (checked) Exclude files from the App_Data folder
• (checked) Include all databases configured in Package/Publish SQL tab - note I do not have any databases configured
• (unchecked) include IIS settings as configured in IIS Express

In the project, I have an App_Data folder setup, primarily to handle application logs.

The behavior I'd like to see (and expect) is the following:

1. On initial deploy to a brand new server, the application is copied and an App_Data folder is created with write permissions assigned for the application.
2. On subsequent deployments, the App_Data folder is ignored because it already exists and the "Exclude files from the App_Data folder" is checked.

However, msdeploy does not appear to do step #1 (step 2 is fine if I create the folder manually). I've been unable to find any documentation on the web besides this unanswered so question that seems to confirm the behavior I see.

How do I get msdeploy to create App_Data and assign permissions on initial deployment in this scenario?

Mentally

Answer 1

Getting App_Data deployed when starting from scratch

@tdykstra got this part right. To get App_Data out there (and ACLs set automatically), I did the following:

1. Adding a placeholder file in App_Data
2. Set the build action to content on the placeholder (my placeholder file has text in it to let people stumbling across it know why it's there).
3. Unchecked "Exclude files from the App_Data folder" on the Package/Publish Web tab of the project properties in VS 2010

This gets my App_Data folder created and ready for use on the server. However, it will result in all my files getting deleted whenever I republish. This is problem #2 in my question above, and pretty closely resembles this other SO question/answer.

Preventing data on the server from being deleted on subsequent publish events

There are two mechanisms in MsDeploy that can get confused (at least I confused them):

1. Excluding files
2. MsDeploy skip rules

These can both be used to solve the problem, depending on the scenario:

1. @tdykstra's solution will likely work if you:
   1. Know the names of the files in App_Data in advance (e.g. a sqllite database)
   2. Have the files included in the App_Data folder in your project
2. The use MsDeploy skip rules to tell MsDeploy to completely skip all deletes on the server for that directory and files in that directory. This solves the problem in all cases, but is much more involved.

Implementing MsDeploy skip rules

To implement skip rules you'll have to abandon the right-click, Deploy option in VS 2010 in favor of right-click, Package, go into a command line, re-jigger a batch file and run a command line). If you're willing to put up with this experience (I am, because I'm automating it all through a CI process), here are the details:

1. Edit the project file and add the following. Note that the AbsolutePath argument is a regular expression, so you can get way fancy:

   <Target Name="AddCustomSkipRules">
       <ItemGroup>
         <MsDeploySkipRules Include="SkipDeleteAppData">
           <SkipAction>Delete</SkipAction>
           <ObjectName>filePath</ObjectName>
           <AbsolutePath>$(_Escaped_PackageTempDir)\\App_Data\\.*</AbsolutePath>
           <XPath>
           </XPath>
         </MsDeploySkipRules>
         <MsDeploySkipRules Include="SkipDeleteAppData">
           <SkipAction>Delete</SkipAction>
           <ObjectName>dirPath</ObjectName>
           <AbsolutePath>$(_Escaped_PackageTempDir)\\App_Data\\.*</AbsolutePath>
           <XPath>
           </XPath>
         </MsDeploySkipRules>
       </ItemGroup>
   </Target>
   

2. Package, do not deploy the project. This will create a zip file and .cmd file in the target directory (defined by "Location where package will be created" on the Package/Publish Web Tab). By default, this is obj\Debug\Package (or obj\Release\Package)
3. Deploy the site using the the resulting command file

In my testing, you must package and run the command file. The project file tweaks will tell msbuild to put the necessary -skip rule into the command file. However, using the "publish" feature straight from VS 2010 doesn't seem to run the command file (see the warning on this walkthrough)...it calls msdeploy directly and doesn't seem to honor the project file skip rules. I believe this is the difference between VS using msbuild -T:Package and msbuild -T:MsDeployPublish to build the project, but I have not tested this.

Finally, the command file isn't quite correct, at least in VS 2010 SP1. There's a great description of what goes wrong in this SO answer, but basically, VS (or maybe the /t:Package target is a better culprit) sets up the command file to publish to the machine without specifying a site. To fix that, you'll need to somehow get "?site=sitename" (probably this is ?site=Default+Web+Site, for a full URL of https://machine:8172/MsDeploy.axd?site=Default+Web+Site) onto the end of the computerName argument.

The problem I had was that the command file (batch file) has a hard time with using site= anything on the command line since it mis-parses the command line argument (even if escaped). I don't see a way around this problem other than modifying the cmd file directly, but for testing I copied the msdeploy.exe output I saw from my failed test run and modified that to call msdeploy.exe directly without the script.

Now that it's working, my intention is to work this into my CI build processes. What I'll be doing for the final solution is:

1. Change my build script to use /T:Package (right now it's /T:MsDeploy)
2. Have a scripted search/replace routine alter the generated cmd deployment script
3. Run the altered deployment script

This really should be easier.

Update

Here's the scripted search/replace routine I've come up with in PowerShell:

(Get-Content "project.deploy.cmd") 
  -replace('^set _ArgComputerName=$'
           ,"set  ArgComputerName=https://server:8172/MsDeploy.axd?Site=Default+Web+Site") 
  | Out-File -Encoding ascii deploy.cmd

Once that is run, deploy.cmd can be called (without the /M option) and it will work as expected.

Mentally

Using VS2017 and MsDeploySkipRules are not really working unless <UseMSDeployExe>true</UseMSDeployExe> is set. – Aggappora

Answer 2

Web Deploy won't create a folder if there are no files to copy to it. One workaround in your scenario would be to not use the Exclude files from the App_Data folder check box, put a dummy file in App_Data (such as a .txt file with nothing in it), and specify file exclusion rules for whatever else you have in the App_Data folder (such as your .sdf file).

On excluding individual files (you can use wildcards), see the first question in the deployment FAQ on MSDN:

http://msdn.microsoft.com/en-us/library/ee942158.aspx#can_i_exclude_specific_files_or_folders_from_deployment

On using the dummy file method for causing a folder to be created, see Making Sure that the Elmah Folder gets Deployed in this tutorial:

http://www.asp.net/web-forms/tutorials/deployment-to-a-hosting-provider/deployment-to-a-hosting-provider-configuring-project-properties-4-of-12

Doughty

This seems like it should work, but I've been mucking with it for an hour now with no success. I can get part 1 completed, but the exclusion of files/directories is not working, with or without wildcards. I'm building through a CI server, so that might be the problem. I'll work on this more in the next day or two and report back. – Mentally

There's a flaw with this approach to solve #2 - in my scenario, I'm want to "exclude" files that are not known to me at build time (e.g. rolling log files with the date in the file name). For this, another approach is needed. I'll add an answer to further elaborate what I've found. – Mentally

You can also use wildcards -- so if your log files all have the same extension, you can exclude all files with that extension. – Doughty

That's true, but only if the log files are actually known to your project file. I tested it without the log files included in the project and it wouldn't work. :( – Mentally

Answer 3

I managed to get it working when using the Publish Web dialog from within Visual Studio. Note: it works for any folder and not only App_Data.

This is the basic .pubxml profile:

<?xml version="1.0" encoding="utf-8"?>
<!--
This file is used by the publish/package process of your Web project. You can customize the behavior of this process
by editing this MSBuild file. In order to learn more about this please visit http://go.microsoft.com/fwlink/?LinkID=208121. 
-->
<Project ToolsVersion="4.0" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
  <PropertyGroup>
<AfterAddIisSettingAndFileContentsToSourceManifest>AddCustomSkipRules</AfterAddIisSettingAndFileContentsToSourceManifest>
    <WebPublishMethod>MSDeploy</WebPublishMethod>
    <LastUsedBuildConfiguration>Local</LastUsedBuildConfiguration>
    <LastUsedPlatform>Any CPU</LastUsedPlatform>
    <SiteUrlToLaunchAfterPublish />
    <ExcludeApp_Data>False</ExcludeApp_Data>
    <MSDeployServiceURL>localhost</MSDeployServiceURL>
    <DeployIisAppPath>SuperCoolAwesomeAppName</DeployIisAppPath>
    <RemoteSitePhysicalPath />
    <SkipExtraFilesOnServer>False</SkipExtraFilesOnServer>
    <MSDeployPublishMethod>InProc</MSDeployPublishMethod>
    <EnableMSDeployBackup>False</EnableMSDeployBackup>
    <UserName />
    <_SavePWD>False</_SavePWD>
    <LaunchSiteAfterPublish>True</LaunchSiteAfterPublish>
  </PropertyGroup>

  <PropertyGroup>
    <UseMsDeployExe>true</UseMsDeployExe>
  </PropertyGroup>

  <Target Name="CreateEmptyFolders">
    <Message Text="Adding empty folders to Files" />
    <MakeDir Directories="$(_MSDeployDirPath_FullPath)\Files\Folder 1" />
    <MakeDir Directories="$(_MSDeployDirPath_FullPath)\Files\Folder 2" />
    <MakeDir Directories="$(_MSDeployDirPath_FullPath)\Files\Folder 3\Test"/>
  </Target>

  <Target Name="AddCustomSkipRules" DependsOnTargets="CreateEmptyFolders">
    <Message Text="Adding Custom Skip Rules" />
    <ItemGroup>
      <MsDeploySkipRules Include="SkipFilesInFilesFolder">
        <SkipAction>Delete</SkipAction>
        <ObjectName>filePath</ObjectName>
        <AbsolutePath>$(_DestinationContentPath)\\Files\\.*</AbsolutePath>
        <Apply>Destination</Apply>
      </MsDeploySkipRules>

      <MsDeploySkipRules Include="SkipFoldersInFilesFolders">
        <SkipAction></SkipAction>
        <ObjectName>dirPath</ObjectName>
        <AbsolutePath>$(_DestinationContentPath)\\Files\\.*\\*</AbsolutePath>
        <Apply>Destination</Apply>
      </MsDeploySkipRules>

    </ItemGroup>
  </Target>

</Project>

---

Here's a detailed post explaining it:

Using MsDeploy publish profile .pubxml to create an empty folder structure on IIS and skip deleting it with MsDeploySkipRules

Euphorbia

Answer 4

Summarizing and simplifying Emil and Leniel answers in a concise one, if you just want to allow App_Data deploy for adds and updates, but prevents deletes, add this to your .pubxml.

<Project>
  ...

  <PropertyGroup>
    <UseMSDeployExe>true</UseMSDeployExe>
    <ExcludeApp_Data>False</ExcludeApp_Data>
  </PropertyGroup>

  <Target Name="AddCustomSkipRules"
          AfterTargets="AddIisSettingAndFileContentsToSourceManifest">
    <Message Text="Adding Custom Skip Rules" />
    <ItemGroup>
      <MsDeploySkipRules Include="SkipDeleteAppData">
        <SkipAction>Delete</SkipAction>
        <ObjectName>filePath</ObjectName>
        <AbsolutePath>App_Data\\.*</AbsolutePath>
      </MsDeploySkipRules>
      <MsDeploySkipRules Include="SkipDeleteAppData">
        <SkipAction>Delete</SkipAction>
        <ObjectName>dirPath</ObjectName>
        <AbsolutePath>App_Data</AbsolutePath>
      </MsDeploySkipRules>
    </ItemGroup>
  </Target>
</Project>

<UseMSDeployExe>true</UseMSDeployExe> is really needed or it will fail complaining Unrecognized skip directive 'skipaction'.

Aggappora