Reorganise index vs Rebuild Index in Sql Server Maintenance plan

Question

In the SSW rules to better SQL Server Database there is an example of a full database maintenance plan: SSW. In the example they run both a Reorganize Index and then a Rebuild Index and then Update Statistics. Is there any point to this? I thought Reorganize Index was a fast but less effective version of Rebuild Index? and that an index rebuild would also update the statistics automatically (on the clustered index at least).

Gnu

Answer 1

Doing a REORGANIZE and then a REBUILD on the same indexes is pointless, as any changes by the REORGANIZE would be lost by doing the REBUILD.

Worse than that is that in the maintenance plan diagram from SSW, it performs a SHRINK first, which fragments the indexes as a side effect of the way it releases space. Then the REBUILD allocates more space to the database files again as working space during the REBUILD operation.

• REORGANIZE is an online operation that defragments leaf pages in a clustered or non-clustered index page by page using little extra working space.

• REBUILD is an online operation in Enterprise editions, offline in other editions, and uses as much extra working space again as the index size. It creates a new copy of the index and then drops the old one, thus getting rid of fragmentation. Statistics are recomputed by default as part of this operation, but that can be disabled.

See Reorganizing and Rebuilding Indexes for more information.

Don't use SHRINK except with the TRUNCATEONLY option and even then if the file will grow again then you should think hard as to whether it's necessary:

sqlservercentral_SHRINKFILE

Kinetics

It's amazing how many online 'authorities' are totally incorrect and misleading, i.e. suggesting you should do a shrink on a database!! – Vanesavanessa

Not true. If you 'reorganize' first, you're potentially compacting the data pages. This is an online operation in all versions of SQL Server and it can be done gradually. A full rebuild requires more resources. By doing the reorganize first and compacting data pages gradually over time, you'll decrease the I/O required by a later rebuild, because it will be reading fewer data pages, causing it to perform less i/o and use less memory. This would be measurable. To say that there's no effect of a reorganize on a subsequent rebuild is nonsense, IMO. – Elseelset

Answer 2

The reorganize and rebuild are different things.

Reorganize: it's a defrag for indexes. Takes the existing index(es) and defragments the existing pages. However if the pages are not in a contiguous manner, they stays like before. Only the content of the pages are changing.

Rebuild: actually it drops the index and rebuilds it from scratch. It means that you will get a completely new index, with defragmented and contiguous pages.

Moreover with rebuild you can change partitioning or file groups, but with reorganize you can defrag not only the whole index, but also only one partition of the index.

The update statistics is automatic on clustered indexes, but not on the non-clustered ones.

Mapp

Right, but is there any use in having both a Reorganize and a Rebuild in the same Maintenance Sub Plan? – Hoarse

Actually, according to Books Online msdn.microsoft.com/en-us/library/ms189858.aspx Reorg DOES reorganize the pages to make they physically contiguous. Here's the exact quote: "Reorganizing an index defragments the leaf level of clustered and nonclustered indexes on tables and views by physically reordering the leaf-level pages to match the logical order (left to right) of the leaf nodes. Having the pages in order improves index-scanning performance. The index is reorganized within the existing pages allocated to it; no new pages are allocated." – Anchusin

@MichaelK.Campbell: Your quote was taken a bit out of context. While ReOrg will re-order the pages, it ONLY reorders them at the lowest level that intermediate level nodes point to them. After a ReOrg all pages across the entire Index are not guaranteed to be contiguous. Here's a better explanation: dba.stackexchange.com/a/36817/6816 – Carol

Answer 3

Before considering maintenance of indexes, it is important to answer two main questions:

1. What is the degree of fragmentation?
2. What is the appropriate action? Reorganize or rebuild?

As described in this article http://solutioncenter.apexsql.com/why-when-and-how-to-rebuild-and-reorganize-sql-server-indexes/, and to help you determine if you should perform index rebuild or index reorganization, please understand the following:

• Index reorganization is a process where the SQL Server goes through existing index, and cleans it up. Index rebuild is a heavy-duty process where index is deleted and then recreated from scratch with entirely new structure, free from all piled up fragments and empty-space pages.

• While index reorganization is a pure cleanup operation which leaves system state as it is without locking-out affected tables and views, the rebuild process locks affected table for the whole rebuild period, which may result in long down-times that could not be acceptable in some environments. With this in mind, it is clear that the index rebuild is a process with ‘stronger’ solution, but it comes with a price – possible long locks on affected indexed tables.

On the other side, index reorganization is a ‘lightweight’ process that will solve the fragmentation in a less effective way – since cleaned index will always be second to the new one fully made from scratch. But reorganizing index is much better from the efficiency standpoint, since it does not lock affected indexed table during the course of operation.

The above mentioned article also explains how to reorganize and rebuild indexes using SSMS, T-SQL (to reorganize/rebuild indexes in a table) and a 3rd party tool called ApexSQL Backup.

Mezoff

when rebuilding index, is it needed to update table statistcs ? especially if there non clustered index ? – Interscholastic

Answer 4

When doing a reorg of an index, if the index is spread across two or more physical files the data will only be defragged within the data file. Pages are not moved from one data file to another.

When the index is in a single file the reorg and reindex will have the same end result.

Some times the reorg will be faster, and some times the reindex will be faster depending on how fragmented the index is. The less fragmented the index then a reorg will be faster, the more fragmented the slower the reorg will be, but the faster a reindex will be.

Hic

Answer 5

Exactly what Biri said. Here is how I would reindex an entire database:

EXEC [sp_MSforeachtable] @command1="RAISERROR('DBCC DBREINDEX(''?'') ...',10,1) WITH NOWAIT DBCC DBREINDEX('?')"

Kelleykelli

Preferred way now is not use Alter Index - learn.microsoft.com/en-us/sql/t-sql/statements/… – Daisie

Answer 6

I Use this SP

CREATE PROCEDURE dbo.[IndexRebuild]
AS 
DECLARE @TableName NVARCHAR(500);
DECLARE @SQLIndex NVARCHAR(MAX);
DECLARE @RowCount INT;
DECLARE @Counter INT;

DECLARE @IndexAnalysis TABLE
    (
      AnalysisID INT IDENTITY(1, 1)
                     NOT NULL
                     PRIMARY KEY ,
      TableName NVARCHAR(500) ,
      SQLText NVARCHAR(MAX) ,
      IndexDepth INT ,
      AvgFragmentationInPercent FLOAT ,
      FragmentCount BIGINT ,
      AvgFragmentSizeInPages FLOAT ,
      PageCount BIGINT
    )

BEGIN
    INSERT  INTO @IndexAnalysis
            SELECT  [objects].name ,
                    'ALTER INDEX [' + [indexes].name + '] ON ['
                    + [schemas].name + '].[' + [objects].name + '] '
                    + ( CASE WHEN (   [dm_db_index_physical_stats].avg_fragmentation_in_percent >= 20
                                    AND [dm_db_index_physical_stats].avg_fragmentation_in_percent < 40
                                  ) THEN 'REORGANIZE'
                             WHEN [dm_db_index_physical_stats].avg_fragmentation_in_percent > = 40
                             THEN 'REBUILD'
                        END ) AS zSQL ,
                    [dm_db_index_physical_stats].index_depth ,
                    [dm_db_index_physical_stats].avg_fragmentation_in_percent ,
                    [dm_db_index_physical_stats].fragment_count ,
                    [dm_db_index_physical_stats].avg_fragment_size_in_pages ,
                    [dm_db_index_physical_stats].page_count
            FROM    [sys].[dm_db_index_physical_stats](DB_ID(), NULL, NULL,
                                                       NULL, 'LIMITED') AS   [dm_db_index_physical_stats]
                    INNER JOIN [sys].[objects] AS [objects] ON (   [dm_db_index_physical_stats].[object_id] = [objects].[object_id] )
                    INNER JOIN [sys].[schemas] AS [schemas] ON ( [objects].[schema_id]  = [schemas].[schema_id] )
                    INNER JOIN [sys].[indexes] AS [indexes] ON (  [dm_db_index_physical_stats].[object_id] = [indexes].[object_id]
                                                          AND  [dm_db_index_physical_stats].index_id = [indexes].index_id
                                                          )
            WHERE   index_type_desc <> 'HEAP'
                    AND [dm_db_index_physical_stats].avg_fragmentation_in_percent > 20
END

SELECT  @RowCount = COUNT(AnalysisID)
FROM    @IndexAnalysis

SET @Counter = 1
WHILE @Counter <= @RowCount 
    BEGIN

        SELECT  @SQLIndex = SQLText
        FROM    @IndexAnalysis
        WHERE   AnalysisID = @Counter

        EXECUTE sp_executesql @SQLIndex

        SET @Counter = @Counter + 1

    END
 GO

and create One Job that execute this SP every week.

Wiltonwiltsey

Doesn't answer the question – Hypodermis

Answer 7

Even better is:

EXEC sp_MSforeachtable 'ALTER INDEX ALL ON ? REINDEX'

or

EXEC sp_MSforeachtable 'ALTER INDEX ALL ON ? REORGANIZE'

Hupp

Answer 8

I researched on web and found some of good articles. At the and i wrote the function and script below which is reorganize, recreate or rebuild all the indexes in a database.

First you may need to read this article to understand why we're not just recreate all indexes.

Second we need a function to build create script for index. So this article may help. Also I'm sharing working function below.

Last step making a while loop to find and organize all indexes in the database. This video is grate example to make this.

Function:

create function GetIndexCreateScript(
    @index_name nvarchar(100)
) 
returns nvarchar(max)
as
begin

declare @Return   varchar(max)

SELECT @Return = ' CREATE ' + 
    CASE WHEN I.is_unique = 1 THEN ' UNIQUE ' ELSE '' END  +  
    I.type_desc COLLATE DATABASE_DEFAULT +' INDEX ' +   
    I.name  + ' ON '  +  
    Schema_name(T.Schema_id)+'.'+T.name + ' ( ' + 
    KeyColumns + ' )  ' + 
    ISNULL(' INCLUDE ('+IncludedColumns+' ) ','') + 
    ISNULL(' WHERE  '+I.Filter_definition,'') + ' WITH ( ' + 
    CASE WHEN I.is_padded = 1 THEN ' PAD_INDEX = ON ' ELSE ' PAD_INDEX = OFF ' END + ','  + 
    'FILLFACTOR = '+CONVERT(CHAR(5),CASE WHEN I.Fill_factor = 0 THEN 100 ELSE I.Fill_factor END) + ','  + 
    -- default value 
    'SORT_IN_TEMPDB = OFF '  + ','  + 
    CASE WHEN I.ignore_dup_key = 1 THEN ' IGNORE_DUP_KEY = ON ' ELSE ' IGNORE_DUP_KEY = OFF ' END + ','  + 
    CASE WHEN ST.no_recompute = 0 THEN ' STATISTICS_NORECOMPUTE = OFF ' ELSE ' STATISTICS_NORECOMPUTE = ON ' END + ','  + 
    -- default value  
    ' DROP_EXISTING = ON '  + ','  + 
    -- default value  
    ' ONLINE = OFF '  + ','  + 
   CASE WHEN I.allow_row_locks = 1 THEN ' ALLOW_ROW_LOCKS = ON ' ELSE ' ALLOW_ROW_LOCKS = OFF ' END + ','  + 
   CASE WHEN I.allow_page_locks = 1 THEN ' ALLOW_PAGE_LOCKS = ON ' ELSE ' ALLOW_PAGE_LOCKS = OFF ' END  + ' ) ON [' + 
   DS.name + ' ] '  
FROM sys.indexes I   
 JOIN sys.tables T ON T.Object_id = I.Object_id    
 JOIN sys.sysindexes SI ON I.Object_id = SI.id AND I.index_id = SI.indid   
 JOIN (SELECT * FROM (  
    SELECT IC2.object_id , IC2.index_id ,  
        STUFF((SELECT ' , ' + C.name + CASE WHEN MAX(CONVERT(INT,IC1.is_descending_key)) = 1 THEN ' DESC ' ELSE ' ASC ' END 
    FROM sys.index_columns IC1  
    JOIN Sys.columns C   
       ON C.object_id = IC1.object_id   
       AND C.column_id = IC1.column_id   
       AND IC1.is_included_column = 0  
    WHERE IC1.object_id = IC2.object_id   
       AND IC1.index_id = IC2.index_id   
    GROUP BY IC1.object_id,C.name,index_id  
    ORDER BY MAX(IC1.key_ordinal)  
       FOR XML PATH('')), 1, 2, '') KeyColumns   
    FROM sys.index_columns IC2   
    --WHERE IC2.Object_id = object_id('Person.Address') --Comment for all tables  
    GROUP BY IC2.object_id ,IC2.index_id) tmp3 )tmp4   
  ON I.object_id = tmp4.object_id AND I.Index_id = tmp4.index_id  
 JOIN sys.stats ST ON ST.object_id = I.object_id AND ST.stats_id = I.index_id   
 JOIN sys.data_spaces DS ON I.data_space_id=DS.data_space_id   
 JOIN sys.filegroups FG ON I.data_space_id=FG.data_space_id   
 LEFT JOIN (SELECT * FROM (   
    SELECT IC2.object_id , IC2.index_id ,   
        STUFF((SELECT ' , ' + C.name  
    FROM sys.index_columns IC1   
    JOIN Sys.columns C    
       ON C.object_id = IC1.object_id    
       AND C.column_id = IC1.column_id    
       AND IC1.is_included_column = 1   
    WHERE IC1.object_id = IC2.object_id    
       AND IC1.index_id = IC2.index_id    
    GROUP BY IC1.object_id,C.name,index_id   
       FOR XML PATH('')), 1, 2, '') IncludedColumns    
   FROM sys.index_columns IC2    
   --WHERE IC2.Object_id = object_id('Person.Address') --Comment for all tables   
   GROUP BY IC2.object_id ,IC2.index_id) tmp1   
   WHERE IncludedColumns IS NOT NULL ) tmp2    
ON tmp2.object_id = I.object_id AND tmp2.index_id = I.index_id   
WHERE I.is_primary_key = 0 AND I.is_unique_constraint = 0 
AND I.[name] = @index_name

return @Return

end

Sql for while:

declare @RebuildIndex Table(
    IndexId int identity(1,1),
    IndexName varchar(100),
    TableSchema varchar(50),
    TableName varchar(100),
    Fragmentation decimal(18,2)
)


insert into @RebuildIndex (IndexName,TableSchema,TableName,Fragmentation)
SELECT 
    B.[name] as 'IndexName', 
    Schema_Name(O.[schema_id]) as 'TableSchema',
    OBJECT_NAME(A.[object_id]) as 'TableName',
    A.[avg_fragmentation_in_percent] Fragmentation
FROM sys.dm_db_index_physical_stats(db_id(),NULL,NULL,NULL,'LIMITED') A 
INNER JOIN sys.indexes B ON A.[object_id] = B.[object_id] and A.index_id = B.index_id  
INNER JOIN sys.objects O ON O.[object_id] = B.[object_id]  
 where B.[name] is not null and B.is_primary_key = 0 AND B.is_unique_constraint = 0 and A.[avg_fragmentation_in_percent] >= 5  

--select * from @RebuildIndex

 declare @begin int = 1
 declare @max int
 select @max = Max(IndexId) from @RebuildIndex
 declare @IndexName varchar(100), @TableSchema varchar(50), @TableName varchar(100) , @Fragmentation decimal(18,2)

 while @begin <= @max
 begin

    Select @IndexName = IndexName from @RebuildIndex where IndexId = @begin
    select @TableSchema = TableSchema  from @RebuildIndex where IndexId = @begin
    select @TableName = TableName  from @RebuildIndex where IndexId = @begin 
    select @Fragmentation = Fragmentation  from @RebuildIndex where IndexId = @begin 

    declare @sql nvarchar(max)
    if @Fragmentation < 31
    begin
        set @sql = 'ALTER INDEX ['+@IndexName+'] ON ['+@TableSchema+'].['+@TableName+'] REORGANIZE WITH ( LOB_COMPACTION = ON )'
        print 'Reorganized Index ' + @IndexName + ' for ' + @TableName + ' Fragmentation was ' + convert(nvarchar(18),@Fragmentation)
    end
    else
    begin
        set @sql = (select dbo.GetIndexCreateScript(@IndexName))
        if(@sql is not null)
        begin
            print 'Recreated Index ' + @IndexName + ' for ' + @TableName + ' Fragmentation was ' + convert(nvarchar(18),@Fragmentation)
        end 
        else
        begin
            set @sql = 'ALTER INDEX ['+@IndexName+'] ON ['+@TableSchema+'].['+@TableName+'] REBUILD PARTITION = ALL WITH (ONLINE = ON)'
            print 'Rebuilded Index ' + @IndexName + ' for ' + @TableName + ' Fragmentation was ' + convert(nvarchar(18),@Fragmentation)
        end
    end

    execute(@sql)


    set @begin = @begin+1

end

Bitter

Answer 9

My two cents... This method follows the spec outlined on tech net: http://technet.microsoft.com/en-us/library/ms189858(v=sql.105).aspx

USE [MyDbName]
GO

SET ANSI_NULLS OFF
GO

SET QUOTED_IDENTIFIER OFF
GO

CREATE PROCEDURE [maintenance].[IndexFragmentationCleanup]
AS
DECLARE @reIndexRequest VARCHAR(1000)

DECLARE reIndexList CURSOR
FOR
SELECT INDEX_PROCESS
FROM (
    SELECT CASE 
            WHEN avg_fragmentation_in_percent BETWEEN 5
                    AND 30
                THEN 'ALTER INDEX [' + i.NAME + '] ON [' + t.NAME + '] REORGANIZE;'
            WHEN avg_fragmentation_in_percent > 30
                THEN 'ALTER INDEX [' + i.NAME + '] ON [' + t.NAME + '] REBUILD with(ONLINE=ON);'
            END AS INDEX_PROCESS
        ,avg_fragmentation_in_percent
        ,t.NAME
    FROM sys.dm_db_index_physical_stats(NULL, NULL, NULL, NULL, NULL) AS a
    INNER JOIN sys.indexes AS i ON a.object_id = i.object_id
        AND a.index_id = i.index_id
    INNER JOIN sys.tables t ON t.object_id = i.object_id
    WHERE i.NAME IS NOT NULL
    ) PROCESS
WHERE PROCESS.INDEX_PROCESS IS NOT NULL
ORDER BY avg_fragmentation_in_percent DESC

OPEN reIndexList

FETCH NEXT
FROM reIndexList
INTO @reIndexRequest

WHILE @@FETCH_STATUS = 0
BEGIN
    BEGIN TRY

        PRINT @reIndexRequest;

        EXEC (@reIndexRequest);

    END TRY

    BEGIN CATCH
        DECLARE @ErrorMessage NVARCHAR(4000);
        DECLARE @ErrorSeverity INT;
        DECLARE @ErrorState INT;

        SELECT @ErrorMessage = 'UNABLE TO CLEAN UP INDEX WITH: ' + @reIndexRequest + ': MESSAGE GIVEN: ' + ERROR_MESSAGE()
            ,@ErrorSeverity = 9 
            ,@ErrorState = ERROR_STATE();

    END CATCH;

    FETCH NEXT
    FROM reIndexList
    INTO @reIndexRequest
END

CLOSE reIndexList;

DEALLOCATE reIndexList;

RETURN 0

GO

Oecd

this is excellent but it does lead to duplicates, meaning reorg or rebuilds can happen multiple times, which should be avoided – Candlenut