SQL server - Delete statement increase the LOG size
Asked Answered
I

2

5

I have a LOGGIN database and it is quite big - 400 GB. It has millions of rows.

I just ran a delete statement which took 2.5 hours and deleted probably millions of rows.

 delete FROM [DB].[dbo].[table]
 where [Level] not in ('info','error')

This is a simple recovery model database. But when I ran the above statement the log files grew to be 800 GB and crashed the server. Why does the LOG file grows for a simple recovery model database?

How can I avoid this in future?

Thanks for your time - RM

Intercurrent answered 13/1, 2014 at 19:0 Comment(2)
#5925971Acculturation
Also see dba.stackexchange.com/questions/56432/… and sqlperformance.com/2013/03/io-subsystem/chunk-deletesBeamends
C
7

I bet you tried to run the whole delete in one transaction. Correct?

Once the transaction is complete, the log space can be reclaimed. Since the transaction never completed, the log file grew until it crashed the server.

Check out my blog entry on How to Delete Large Data.

http://craftydba.com/?p=3079

The key to the solution is the following, SIMPLE recover mode, DELETE in small batches, take FULL backup at end of purge. Select the recovery model that you want at the end.

Here is some sample code to help you on your way.

--
-- Delete in batches in SIMPLE recovery mode
--

-- Select correct db
USE [MATH]
GO

-- Set to simple mode
ALTER DATABASE [MATH] SET RECOVERY SIMPLE;
GO

-- Get count of records
SELECT COUNT(*) AS Total FROM [MATH].[dbo].[TBL_PRIMES];
GO

-- Delete in batches
DECLARE @VAR_ROWS INT = 1;
WHILE (@VAR_ROWS > 0)
BEGIN
    DELETE TOP (10000) FROM [MATH].[dbo].[TBL_PRIMES]; 
    SET @VAR_ROWS = @@ROWCOUNT;
    CHECKPOINT;
END;
GO

-- Set to full mode
ALTER DATABASE [MATH] SET RECOVERY FULL;
GO

Last but not least, if the amount of remaining data after the delete is real small, it might be quicker to do the following.

1 - SELECT * INTO [Temp Table] WHERE (clause = small data).
2 - DROP [Original Table].
3 - Rename [Temp Table] to [Original Table].
4 - Add any constraints or missing objects.

The DROP table action does not LOG all the data being removed.

Sincerely,

John

Conservator answered 13/1, 2014 at 20:0 Comment(5)
For your last but not least section, you could also truncate the table and then insert back the rows you wanted to keep and temporarily stored in [Temp Table]. Still about the same amount of work, especially if there are constraints etc., but the bonus would be you don't have to re-establish permissions.Beamends
@AaronBertrand this is no option with fk/on-delete-cascadeFreezing
@Freezing Of course not. But you can drop and re-create those constraints depending on what's more important to you. In 20+ years working with SQL Server across a wide variety of implementations, I've yet to be forced into FKs that cascade.Beamends
I'm a bit scared of switching to SIMPLE recovery mode from FULL on a production database. What are the risks of making this switch?Jacobsohn
Hi Rebecca, going to simple mode breaks the restore chain for a database with full recovery. Make sure you take a full backup before making changes. Change to simple mode. Do the clean up. Take another full backup and change to full mode. This technique is only valuable for very large tables. You can use the code as is w/o the recovery mode changes. Just increase the frequency of your log backup schedule to reduce log file growth. If you do not care about log file growth, use as is. I would make sure the growth rate is set to fixed size to prevent VLFS. - JohnConservator
V
0

Consider using an open-source PowerShell Module sqlsizer-msql.

It's available on GitHub and it's published under MIT license: https://github.com/sqlsizer/sqlsizer-mssql

I think could help you with your task. It has "slow delete" feature.

Velours answered 20/9, 2022 at 20:46 Comment(1)
This does not provide an answer to the question. Once you have sufficient reputation you will be able to comment on any post; instead, provide answers that don't require clarification from the asker. - From ReviewThaler

© 2022 - 2024 — McMap. All rights reserved.