How can I conditionally include large scripts in my ssdt post deployment script?
Asked Answered
W

5

5

In our SSDT project we have a script that is huge and contains a lot of INSERT statements for importing data from an old system. Using sqlcmd variables, I'd like to be able to conditionally include the file into the post deployment script.

We're currently using the :r syntax which includes the script inline:

IF '$(ImportData)' = 'true'
BEGIN
  :r .\Import\OldSystem.sql
END

This is a problem because the script is being included inline regardless of whether $(ImportData) is true or false and the file is so big that it's slowing the build down by about 15 minutes.

Is there another way to conditionally include this script file so it doesn't slow down the build?

Wilbertwilborn answered 10/5, 2016 at 16:1 Comment(3)
:R command as well as all others SQLCMD commans do not work within a conditional block - they execute always. The workaround is to wrap your old system code with IF '$(ImportData)' = 'true' withing the OldSystem.sql fileButyraldehyde
https://mcmap.net/q/319399/-conditional-logic-in-postdeployment-sql-script-using-sqlcmdButyraldehyde
For performance, have you considered bulk inserting rather than INSERTing?Billmyre
W
0

I ended up using a mixture of our build tool (Jenkins) and SSDT to accomplish this. This is what I did:

  1. Added a build step to each environment-specific Jenkins job that writes to a text file. I either write a SQLCMD command that includes the import file or else I leave it blank depending on the build parameters the user chooses.
  2. Include the new text file in the Post Deployment script via :r.

That's it! I also use this same approach to choose which pre and post deploy scripts to include in the project based on the application version, except that I grab the version number from the code and write it to the file using a pre-build event in VS instead of in the build tool. (I also added the text file name to .gitignore so it doesn't get committed)

Wilbertwilborn answered 8/2, 2017 at 15:11 Comment(0)
S
7

Rather than muddy up my prior answer with another. There is a special case with a VERY simple option.

Create separate SQLCMD input files for each execution possibility. The key here is to name the execution input files using the value of your control variable.

So, for example, your publish script defines variable 'Config' which may have one of these values: 'Dev','QA', or 'Prod'.

Create 3 post deployment scripts named 'DevPostDeploy.sql', 'QAPostDeploy.sql' and 'ProdPostDeploy.sql'.

Code your actual post deploy file like this:

:r ."\"$(Config)PostDeploy.sql

This is very much like the build event mechanism where you overwrite scripts with appropriate ones except you don't need a build event. But you are dependent upon naming your scripts very specifically.

Six answered 13/5, 2016 at 20:38 Comment(6)
My database is deployed twice with different SQLCMD properties, that are used in this way. Only the value passed in with the second deploy is used in both deployments. It's almost like the value of the first one is overwritten. Only they are done in sequence... The target connection for the first deployment is the right one.Thigpen
Fair warning - if you are still stuck with VS2013, the file included via this method will not necessarily match the "publish" dialog. It appears that it will always include the file based on the value of the variable specified in the Project properties on the SQLCMD variables tab. I have not tested with more recent versions of Visual Studio.Newfashioned
Many thanks Nathan for this comment, I am experiencing the same behaviour and could not understand why the same deploy file was being included regardless of what values I entered. This seems like a nice solution but it does not work in all circumstances.Synn
This no longer seems to work, I've tried using sqlcmd variables set in the project settings but it just comes up as empty in the generated script. For example $(SqlCmdVariable1), when used in a path as :r ."\"$(SqlCmdVariable1)$(InlineSetVar1).. The path it takes is everything except the SqlCmdVariable.. the InlineSetVar1 I setup in the postdeploy file works fine such as :setvar InlineSetVar1 "_Something.Sql"Theomachy
Thanks @JARRRRG. It might be useful to mention the version in which you noticed it no longer works.Six
@Six this was in a new Sql Database project in vs2019 targeting azure sql db.Theomachy
C
1

The scripts referenced using :r are always included. You have a couple of options but I would first verify that if you take the script out it improves the performance to where you want it to get to.

The simplest approach is to just keep it outside of the whole build process and change your deploy process so it becomes a two step thing (deploy DAC then deploy script). The positives of this are you can do things outside of the ssdt process but the negatives are you don't get things like auto disabling of constraints on tables changing in the deployment.

The second way is to not include the script in the deploy when you build but create an AfterBuild msbuild task that adds the script as a post deploy script in the dacpac. The dacpac is a zip file so you can use the .net packaging Api to add a part called postdeploy.sql which will then be included in the deployment process.

Both of these ways mean you lose verification so you might want to keep it in a separate ssdt project which has a "same database" reference to your main project, it will slow down the build when it changes but should be quick the rest of the time.

Crier answered 10/5, 2016 at 17:13 Comment(0)
S
0

Here is the way I had to do it.

1) Create a dummy post-deploy script.

2) Create build configurations in your project for each deploy scenario.

3) Use a pre-build event to determine which post deploy configuration to use. You can either create separate scripts for each configuration or dynamically build the post-deploy script in your pre-build event. Either way you base what you do on the value of $(configuration) which always exists in a build event.

If you use separate static scripts, your build event only needs to copy the appropriate static file, overwriting the dummy post-deploy with whichever script is useful in that deploy scenario.

In my case I had to use dynamic generation because the decision about which scripts to include required knowing the current state of the database being deployed to. So I used the configuration variable to tell me which environment was being deployed to and then used an SQLCMD script with :OUT set to my Post-Deploy script location. Thus my pre-build script would then write the post-deploy script dynamically.

Either way, once build completed and the normal deploy process started the Post-Deploy script contained exactly the :r commands that I wanted.

Here's an example of the SQLCMD script I invoke in pre-build.

:OUT .\Script.DynamicPostDeployment.sql

PRINT ' /*';
PRINT '     DO NOT MANUALLY MODIFY THIS SCRIPT.                                               ';
PRINT '                                                                                       ';
PRINT '     It is overwritten during build.                                                   ';
PRINT '     Content IS based on the Configuration variable (Debug, Dev, Sit, UAT, Release...) ';
PRINT '                                                                                       ';
PRINT '     Modify Script.PostDeployment.sql to effect changes in executable content.         ';
PRINT ' */';
PRINT 'PRINT ''PostDeployment script starting at''+CAST(GETDATE() AS nvarchar)+'' with Configuration = $(Configuration)'';';
PRINT 'GO';
IF '$(Configuration)' IN ('Debug','Dev','Sit')
BEGIN
    IF (SELECT IsNeeded FROM rESxStage.StageRebuildNeeded)=1
    BEGIN
        -- These get a GO statement after every file because most are really HUGE
        PRINT 'PRINT ''ETL data was needed and started at''+CAST(GETDATE() AS nvarchar);';
        PRINT '                                                  ';
        PRINT 'EXEC iESxETL.DeleteAllSchemaData ''pExternalETL'';';
        PRINT 'GO';
        PRINT ':r .\PopulateExternalData.sql         ';
....
Six answered 13/5, 2016 at 15:36 Comment(0)
W
0

I ended up using a mixture of our build tool (Jenkins) and SSDT to accomplish this. This is what I did:

  1. Added a build step to each environment-specific Jenkins job that writes to a text file. I either write a SQLCMD command that includes the import file or else I leave it blank depending on the build parameters the user chooses.
  2. Include the new text file in the Post Deployment script via :r.

That's it! I also use this same approach to choose which pre and post deploy scripts to include in the project based on the application version, except that I grab the version number from the code and write it to the file using a pre-build event in VS instead of in the build tool. (I also added the text file name to .gitignore so it doesn't get committed)

Wilbertwilborn answered 8/2, 2017 at 15:11 Comment(0)
M
0

Since using :r in pre-deploy/post-deploy scripts to select an environment hasn't been working in SSDT for a while and this is among the top answers on Google for this, I thought I'd leave this here.

Credit to commenter "Bulgom" on https://ntsblog.homedev.com.au/index.php/2013/03/13/sqlcmd-variables-linked-server-connection-intellisense/ (Visible at the time of writing)

Excerpt:

In SQLCMD Variable config, replace LinkedServer].[RemoteDb by LinkedServer”.”RemoteDb In scripts replace [$(RemotDB)] by “$(RemoteDb)”.

I have tested with VS 17.6.5 SSDT, SSMS 19.1.56.0, localdb 15.0.4153.1, and SQL Server 15.0.2101.7. All are happy to interpret "SQLLinkedServer"."RemoteDB".[dbo].[Stored procedure] and "RemoteDB".[dbo].[Stored procedure] correctly.

Looks like this in the project configuration: SQLCMD Variables config in project.

Looks like this in the local publish profile SQLCMD Variable config in publish profile

Comes out like so in the publish script:

SQLCMD Variable set in publish script

The code in the publish script making use of the variable

SQLCMD Variable used in publish script

Results are thus when doing Right-Click -> Modify in the DB instance:

Result in live database

Note that I get an angry squiggle from intellisense in SSMS, but for overall usability, this is far better than reminding everyone they need to delete the extra closing square bracket.

Mizzenmast answered 25/8, 2023 at 12:47 Comment(0)

© 2022 - 2025 — McMap. All rights reserved.