Auto increment even and odd for two databases for synchronization without affecting auto increment property
Asked Answered
P

5

5

Need quick help. I am having my database with bigint auto increment property. I have database on two location's which need to synchronized. As bigint is not good choice for synchronization because of possibility primary key replica on different sites. I can not move on with GUID because for that i need to change my code as well as database which is not possible for me.

Right now i have two location only for database, so i think if can make my primary key auto increment to be always even at one location and to be odd at other location. It can solve my issue in quick manner.

How can i do it using computed column specification or by any other way. For synchronization i am using Microsoft sycn framework.

If i use identity(1,2) A server or identity(2,2) B server after synchronization it disturbs next increment value. For example if at A server max id 3 and at B server current id is 4. After sync max id on A server will be now 4. I want new id on A server should be 5 only but in reality it inserts 6. How can i solve this issue

Pontiff answered 11/3, 2015 at 9:33 Comment(5)
dev.mysql.com/doc/refman/5.0/en/…Serinaserine
What are you using to synchronize the two databases? Replication has built in options to manage identity ranges and you won't have stuck yourself with later issues if you want/need a third location.Rettarettig
for that i need to spend some time as i am done with synchronization now i only need is different primary key and i am lacking in time to fulfill my motivePontiff
I think you need a trigger for gaining data with making an Identity, And suggest you to save time of any insert to a specific field by getDate().Siloum
What version of SQL Server do you use? Add a corresponding tag to the question. If you use SQL Server 2012 or later it has SEQUENCE, which gives finer control over generated numbers.Intercolumniation
R
4

Big Edit: (Much better) Option 1:

(Additional note: @VladimirBaranov mentioned this in the comments, and I missed it, but here's a fleshing out of how to use a SEQUENCE in this situation anyway)

My original idea is further down this answer, and would still be potentially viable, but I think this newer option will fit the bill exactly for however many servers you need. It was bothering me that I knew there was a right way to do this in TSQL, and I could not remember what it was. My brain finally dredged it up today: SEQUENCE. SQL Server 2012 and 2014 allow you to define a sequence to generate a series of numbers for use in your tables:

CREATE SEQUENCE oddNums
  START WITH 1
  INCREMENT BY 2;
GO

CREATE SEQUENCE evenNums
  START WITH 0
  INCREMENT BY 2;
GO

Then instead of AUTO INCREMENTing your PK's, give them a DEFAULT value from the SEQUENCE (these are the tables from the fiddle linked below):

CREATE TABLE oddMirror (
 [id] int PRIMARY KEY DEFAULT NEXT VALUE FOR oddNums, 
 [data] varchar(7)
);

CREATE TABLE evenMirror (
 [id] int PRIMARY KEY DEFAULT NEXT VALUE FOR evenNums, 
 [data] varchar(7)
);

These sequences are totally unaffected by merges, and will continue to generate odd or even numbers forever no matter what the latest PK in the table is.

Here is a SQLFiddle of this in action.

Note that you can't define the column as IDENTITY if you do this (because of the DEFAULT clause), so you will have to be careful about inserting into your id column, but otherwise this should be about as straightforward as it gets.

This could be done with as many servers as you want, just adjust how much each SEQUENCE increments by and where it starts from, but you would have a hard (not impossible) time adding additional servers to the mix once your SEQUENCEs were defined.

Also, here is an MSDN blog that discusses alternative strategies for simulating a SEQUENCE on prior versions of SQL Server.

(Not as good) Option 2:

(Note: this is my original answer) I have been playing with this some this evening, and depending on how you have things set up, I think you could get away with reseeding the table on each server after the sync is done, based on the current highest id in the table. You would just have to do it slightly differently for each server to keep new ids odd on one and even on the other.

So you have:

CREATE TABLE oddMirror
(id INT NOT NULL IDENTITY(1,2),
data NVARCHAR(10))
GO

and

CREATE TABLE evenMirror
(id INT NOT NULL IDENTITY(2,2),
data NVARCHAR(10)
GO

After you sync the two tables, you don't know if the current identity seed is odd or even, so you need to reset it on each table to the correct "next" value for the server. So, on oddMirror:

DECLARE @maxId INT
DECLARE @newSeed INT
SET @maxId = (SELECT MAX(id) FROM oddMirror)
SET @newSeed = (SELECT CASE WHEN @maxId % 2 = 1 THEN @maxId ELSE @maxId -1 END)

DBCC CHECKIDENT('dbo.oddMirror', RESEED, @newSeed)
GO

And an almost identical process on evenMirror:

DECLARE @maxId INT
DECLARE @newSeed INT
SET @maxId = (SELECT MAX(id) FROM evenMirror)
SET @newSeed = (SELECT CASE WHEN @maxId % 2 = 0 THEN @maxId ELSE @maxId -1 END)

DBCC CHECKIDENT('dbo.evenMirror', RESEED, @newSeed)
GO

So basically, on oddMirror we are saying, "Get the current max id. if it's odd, don't change it, but if it's even, back it up by one."

Then do the same thing on 'evenMirror', except check whether the max id is even instead of odd.

So as an example, take this data:

oddMirror
1,"one"
3,"three"
5,"five"

and

evenMirror
2,"two"
4,"four"
6,"six"
8,"eight"

(notice that evenMirror has more rows)

After a sync, each table would look like this:

oddMirror
1,"one"
2,"two"
3,"three"
4,"four"
5,"five"
6,"six"
8,"eight"
--evenMirror looks the same as this now

Running things through the above queries:

MAX(id) on oddMirror is 8. 8 % 2 = 0, so set @newSeed = 8 - 1 = 7, meaning the next row in oddMirror will get id = 9.

MAX(id) on evenMirror is also 8, but the query is slightly different. 8 % x = 0 so set @newSeed = 8, meaning the next row in 'evenMirrorwill getid = 10`

id = 7 will be skipped in this scenario, but I'm guessing that's not a tremendous concern.

If you then queried:

INSERT INTO oddMirror (data) VALUE ("data")
GO

INSERT INTO evenMirror (data) VALUE ("otherData")
GO

Tables would look like this:

oddMirror
1,"one"
2,"two"
3,"three"
4,"four"
5,"five"
6,"six"
8,"eight"
9,"data"

and

evenMirror
1,"one"
2,"two"
3,"three"
4,"four"
5,"five"
6,"six"
8,"eight"
10,"otherData"

This could theoretically be expanded out to accommodate more number of servers by changing what modulo you take and adding additional WHENs to the CASE statement for each possibility, although that would certainly get cumbersome to maintain. But, we already knew that the right solution (GUIDs) is not available here, and if you're reading this far, the next best solution (SEQUENCEs) may not be available, so whatever we come up with is inevitably going to be cumbersome.

The biggest drawback to this approach is that the tables must be locked until the sync process is complete. If a write comes in before sync is complete and ids are reseeded, there will almost certainly be a collision. If the tables are not written to very often, or you are already locking them for syncing, or if you have a significant "dead" spot in your daily cycle (like 3-4 a.m. or something) where this could be done without serious disruption, it may not be that big of a deal, but only you will know how viable this is.

So, your setup may or may not make this possible, but I have been playing with this quite a bit tonight in my sandbox db, and it seems to work well to make sure that new id's are always odd in one db and always even in the other.

Rone answered 15/3, 2015 at 0:59 Comment(6)
I asked @Hot Cool Stud what version of SQL Server he uses, but he never replied. I guess this problem is not really critical for him.Intercolumniation
@VladimirBaranov Who knows, maybe not. Sometimes I can't resist an interesting problem, though. It was really bothering me that I couldn't remember the right schema construct to use for this...Rone
I mentioned SEQUENCE for SQL Server 2012+ in the comment under the question on the 14th of March. It must be hard to notice such things in the comments. It is good that you arrived to the same idea yourself. You tend to remember solutions that you found yourself much more vividly than if you just read about them somewhere. I agree with you that for SQL Server 2012+ SEQUENCE is a very good solution.Intercolumniation
@VladimirBaranov Totally missed that, didn't mean to step on your toes ;]Rone
I believe option2 is risky because synchronization process can be long. Therefore, an insertion can happen while the synchronization is in progress. In this case, the reseed is done too late. Of course, we could fix this by locking the table while synchronizing but then others query will be left waiting...Harbert
@Harbert All of this is true. Option 2 is definitely dependent on what your sync process looks like, how long it takes, how often it happens, and whether you need to be able to write new records to the tables during the sync process, not to mention what sort of permissions you have on the server. I would say that if you have them available, SEQUENCEs are hands-down the right solution to this problem.Rone
I
3

Here is a very simple solution, but it will work only for two servers. It can't be easily extended for more servers.

The good thing about it is that it doesn't use CHECKIDENT to reseed the tables and you don't need to worry about simultaneously running transactions to get the accurate MAX ID to feed into CHECKIDENT.

Also, MSDN warns that identity property on a column does not guarantee the following:

Consecutive values after server restart or other failures – SQL Server might cache identity values for performance reasons and some of the assigned values can be lost during a database failure or server restart. This can result in gaps in the identity value upon insert. If gaps are not acceptable then the application should use its own mechanism to generate key values. Using a sequence generator with the NOCACHE option can limit the gaps to transactions that are never committed.

If you choose a solution that is based on reseeding identity using CHECKIDENT you'd better double check that it works correctly in such cases.

Also, to run CHECKIDENT you may need specific permissions:

Caller must own the table, or be a member of the sysadmin fixed server role, the db_owner fixed database role, or the db_ddladmin fixed database role.

Solution

My main idea is that on the first server you use IDENTITY(1,1) and on the second server you use IDENTITY(-1,-1). Rather than trying to make IDs odd and even they will be positive and negative.

Here is a script that proves that it works as intended without any extra work.

-- Sample data
CREATE TABLE #T1 (ID bigint IDENTITY(1,1), V1 int);
CREATE TABLE #T2 (ID bigint IDENTITY(-1,-1), V2 int);

INSERT INTO #T1 VALUES (11);
INSERT INTO #T1 VALUES (12);
INSERT INTO #T1 VALUES (13);
INSERT INTO #T1 VALUES (14);

INSERT INTO #T2 VALUES (21);
INSERT INTO #T2 VALUES (22);
INSERT INTO #T2 VALUES (23);

SELECT * FROM #T1;
SELECT * FROM #T2;

We start with this sample data in our tables:

#T1
ID  V1
1   11
2   12
3   13
4   14

#T2 
ID  V2
-1  21
-2  22
-3  23

Perform the sync

-- Insert into T1 new values from T2
SET IDENTITY_INSERT #T1 ON;

MERGE INTO #T1 AS Dst
USING
(
    SELECT ID, V2
    FROM #T2
) AS Src
ON Dst.ID = Src.ID
WHEN NOT MATCHED BY TARGET
THEN INSERT (ID, V1)
VALUES (Src.ID, Src.V2);

SET IDENTITY_INSERT #T1 OFF;

-- Insert into T2 new values from T1
SET IDENTITY_INSERT #T2 ON;

MERGE INTO #T2 AS Dst
USING
(
    SELECT ID, V1
    FROM #T1
) AS Src
ON Dst.ID = Src.ID
WHEN NOT MATCHED BY TARGET
THEN INSERT (ID, V2)
VALUES (Src.ID, Src.V1);

SET IDENTITY_INSERT #T2 OFF;

SELECT * FROM #T1;
SELECT * FROM #T2;

Result of the sync - two identical tables

#T1
ID  V1
1   11
2   12
3   13
4   14
-1  21
-2  22
-3  23


#T2
ID  V2
-1  21
-2  22
-3  23
1   11
2   12
3   13
4   14

Insert more data to check how identity works after the sync

-- Insert more data into T1 and T2
INSERT INTO #T1 VALUES (15);
INSERT INTO #T1 VALUES (16);

INSERT INTO #T2 VALUES (24);
INSERT INTO #T2 VALUES (25);
INSERT INTO #T2 VALUES (26);

SELECT * FROM #T1;
SELECT * FROM #T2;

-- Clean up
DROP TABLE #T1;
DROP TABLE #T2;

Generated identities after the sync

#T1
ID  V1
1   11
2   12
3   13
4   14
-1  21
-2  22
-3  23
5   15
6   16

#T2
ID  V2
-1  21
-2  22
-3  23
1   11
2   12
3   13
4   14
-4  24
-5  25
-6  26

You can see that new identities in T1 continue to be positive and new identities in T2 continue to be negative.

Intercolumniation answered 15/3, 2015 at 3:30 Comment(0)
S
1

You have two source table for inserting to one destination table :
So I suggest you to do these :

  • [pkId]: Have an identity field in destination table as PK.
  • [src]: Add an integer -or any other as you want- field and update it from 1st. source data by1and for the 2nd one by2`.
  • [Id]: You have also a field that comes from sources.
  • [nId]: Add a bigint field that is null.

Now use this query to have your enumerated Id in nId field:

Update <table> 
Set nId = isnull((select count(ti.*) from <table> as ti where ti.pkId < <table>.pkId), 0) + 1

For running this query after any inserts you can use triggers.

I think with this solution you have all data you need.

Edit some results:

pkId | src | Id | nId before query | nId after query
-----+-----+----+------------------+--------------------
 1   | 1   | 1  | null             | 1
 2   | 2   | 1  | null             | 2
 3   | 1   | 2  | null             | 3
 5   | 2   | 2  | null             | 4
 6   | 2   | 3  | null             | 5
 8   | 1   | 3  | null             | 6
Siloum answered 14/3, 2015 at 9:4 Comment(4)
can you please explain you query what you are trying to do in that.Pontiff
As pkId is auto-increment and you need a specific unique field -nId-; With that query you can generate it, for first all nIds are null, for first row -pkId = 1- => nId = 1 and ... , and all are exclude the source table where in pkid = 2, Id = 1, src = 2 => nId = 2 and etc.Siloum
Usually this method would bind Id and Source together into the primary key. Otherwise you still have the same issues. You could tweak this with a computed column to get a single unique ID. The following would work for two servers, you'd have to tweak it for 3+. CASE WHEN (src = 2) THEN -1 * Id ELSE Id END;Apeak
this con work for n servers too, just at inserting time set src = n and in another table store srcId and servername for addition.Siloum
D
1

An option, have one server set with identity(1,1), and the other set with identity(-1,-1). Identities will not overlap, and copying data from one server to the other would not affect the "next" id or reseeding.

Doesn't work for more than two servers, of course.

Drees answered 19/3, 2015 at 21:9 Comment(0)
S
0

I think that the even/odd approach is making this very hard. Also, as you add rows to each node, you will have page-split issues, especially if your PK is the clustered index.

Are these tables replicated using peer-to-peer replication or are they manually synchronized? The page split issues would come into play if there is replication involved.

Why not use ranges of numbers for each node? 1-X for node 1, and X+1-Y for Node#2? Estimate your row volume and set the ranges so large that overlap won't occur.

Example for BIGINT:

Node 1: 1-200000000000 (200 billion rows) Node 2: 200000000001-600000000000 (400 billion rows)

Leaving 600000000001 and up for future use. Caveat, the Identity does not have a maximum value, you need to police that manually.

To set the identity value to the correct number use DBCC CHECKIDENT with the RESEED option. This would also work if you're married to the even/odd scenario.

This also have the advantage of not page splitting once per insert, especially if the activity per node is not evenly balanced.

Solander answered 21/3, 2015 at 16:22 Comment(0)

© 2022 - 2024 — McMap. All rights reserved.