Hibernate, @SequenceGenerator and allocationSize

Question

We all know the default behaviour of Hibernate when using @SequenceGenerator - it increases real database sequence by one, multiple this value by 50 (default allocationSize value) - and then uses this value as entity ID.

This is incorrect behaviour and conflicts with specification which says:

allocationSize - (Optional) The amount to increment by when allocating sequence numbers from the sequence.

To be clear: I do not bother about gaps between generated IDs.

I care about IDs that are not consistent with underlying database sequence. For example: any other application (that e.g. uses plain JDBC) may want to insert new rows under IDs obtained from sequence - but all those values may be already used by Hibernate! Madness.

Do somebody know any solution to this problem (without setting allocationSize=1 and thus degrading performance)?

EDIT:
To make things clear. If last inserted record had ID = 1, then HB use values 51, 52, 53... for its new entities BUT at the same time: sequence's value in database will be set to 2. Which can easily leads to errors when other applications are using that sequence.

On the othe hand: specification says (in my understanding) that database sequence should have been set to 51 and in the meantime HB should use values from range 2, 3 ... 50

UPDATE:
As Steve Ebersole mentioned below: the behaviour described by me (and also the most intuitive for many) can be enabled by setting hibernate.id.new_generator_mappings=true.

Thanks all of You.

UPDATE 2:
For future readers, below you can find a working example.

@Entity
@Table(name = "users")
public class User {

    @Id
    @GeneratedValue(strategy = GenerationType.SEQUENCE, generator = "USERS_SEQ")
    @SequenceGenerator(name = "USERS_SEQ", sequenceName = "SEQUENCE_USERS")
    private Long id;
}

persistence.xml

<persistence-unit name="testPU">
  <properties>
    <property name="hibernate.id.new_generator_mappings" value="true" />
  </properties>
</persistence-unit>

Recognize

"without setting allocationSize=1 and thus degrading performanc" why it degrade performance is you set it to 1? – Floatplane

@Floatplane see may comment below :-) This is because every save needs to query database for next value of the sequence. – Recognize

Thank you was facing the same issue. At first I was adding allocationSize = 1 at every @SequenceGenerator. Using hibernate.id.new_generator_mappings=true prevent that. Although JPA still query the database to get the id for each insert ... – Vesicant

With SequenceGenerator Hibernate will query the database only when amount of IDs specified by allocationsize runs out. If you set up allocationSize = 1 then it's the reason why Hibernate query the DB for each insert. Change this value, and you are done. – Recognize

Thanks! the hibernate.id.new_generator_mappings setting is really important. I would hope it is the default setting that I do not have to spend so much time research why the id number goes wild. – Spavined

For the working example to work indeed, do I need to create my sequence with the parameter "INCREMENT BY 50"? Or is this handled automatically by Hibernate? – Theodor

set the 'hibernate.id.sequence.increment_size_mismatch_strategy' to override the default value in the code, then all setting will read from db – Eddra

from Hibernate 5.0 hibernate.id.new_generator_mappings is set by default to true Hibernate_User_Guide – Fash

Answer 1

To be absolutely clear... what you describe does not conflict with the spec in any way. The spec talks about the values Hibernate assigns to your entities, not the values actually stored in the database sequence.

However, there is the option to get the behavior you are looking for. First see my reply on Is there a way to dynamically choose a @GeneratedValue strategy using JPA annotations and Hibernate? That will give you the basics. As long as you are set up to use that SequenceStyleGenerator, Hibernate will interpret allocationSize using the "pooled optimizer" in the SequenceStyleGenerator. The "pooled optimizer" is for use with databases that allow an "increment" option on the creation of sequences (not all databases that support sequences support an increment). Anyway, read up about the various optimizer strategies there.

Benedikta

Thanks Steve! The best answer. Also your other post was helpful. – Recognize

I also noticed that you are co-author of org.hibernate.id.enhanced.SequenceStyleGenerator. You suprised me. – Recognize

Surprised you how? I am the lead developer of Hibernate. I have wrote/co-wrote many Hibernate classes ;) – Benedikta

Just for the record. DB sequence Increment should be avoided to prevent large gaps. DB sequence is multiplied with allocationSize when ID cache runs out.More details #5346647 – Holp

One way to change the "optimizer" used globally is to add something like this to your hibernate options: serviceBuilder.applySetting("hibernate.id.optimizer.pooled.preferred", LegacyHiLoAlgorithmOptimizer.class.getName()); Instead of the LegacyHiLoAlgorithOptimizer you can pick any optimizer class, and it will become default. This should make it easier to keep the behavior you want as default without changing all annotations. In addition, watch out with the "pooled" and "hilo" optimizers: these give odd results when your sequence value starts at 0 causing negative ID's. – Tribalism

Simply not true @OlcayTarazan. You should always prefer to use blocks of values from a sequence (allocation > 1). First, allocationSize = 1 means Hibernate has to hit the database every single time you ask it to generate an id -> not ideal. Second, the post you link to was true in older versions of Hibernate, but we no longer use hilo by default. Can you end up with gaps when using allocation > 0? Sure, if your app crashes, etc. You can have your fully sequential values no matter what; I'll take the performance – Benedikta

Answer 2

allocationSize=1 is a micro optimization before getting query Hibernate tries to assign value in the range of allocationSize and so try to avoid querying database for sequence. But this query will be executed every time if you set it to 1. This hardly makes any difference since if your data base is accessed by some other application then it will create issues if same id is used by another application meantime .

Next generation of Sequence Id is based on allocationSize.

By default it is kept as 50 which is too much. It will also only help if your going to have near about 50 records in one session which are not persisted and which will be persisted using this particular session and transation.

So you should always use allocationSize=1 while using SequenceGenerator. As for most of underlying databases sequence is always incremented by 1.

Telium

Nothing to do with performance? Are you really sure? I has been taught that with allocationSize=1 Hibernate on every save operation needs to do the trip to the database in order to obtain new ID value. – Recognize

It is a micro optimization before getting query Hibernate tries to assign value in the range of allocationSize and so try to avoid querying database for sequence. But this query will be executed every time if you set it to 1. This hardly makes any difference since if your data base is accessed by some other application then it will create issues if same id is used by another application meantime – Telium

And yes, it is completely application specific whether an allocation size of 1 has any real performance impact. In a micro benchmark, of course, it is always going to show up as a huge impact; thats the problem with most benchmarks (micro or otherwise), they simply are not realistic. And even if they are complex enough to be somewhat realistic, you still have to look at how close the benchmark is to your actual application to understand how applicable the benchmark results are to the results you would see in your app. Long story short.. test it for yourself – Benedikta

OK. Everything is application specific, isn't it! In case your application is a read only application then the impact of using allocation size 1000 or 1 is absolutely 0. On the other hand things like these are best practices. If you do not respect the best practices they gather up and the combined impact will be your application becomes sluggish. Another example would be starting a transaction when you absolutely do not need one. – Ramp

Answer 3

I would check the DDL for the sequence in the schema. JPA Implementation is responsible only creation of the sequence with the correct allocation size. Therefore, if the allocation size is 50 then your sequence must have the increment of 50 in its DDL.

This case may typically occur with the creation of a sequence with allocation size 1 then later configured to allocation size 50 (or default) but the sequence DDL is not updated.

Ramp

You are misunderstanding my point. ALTER SEQUENCE ... INCREMENTY BY 50; won't solve anything, because problem still remains the same. Sequence value still doesn't reflect real entities IDs. – Recognize

Please share a test case so that we can better understand the problem here. – Ramp

Test case? Why? Question posted by me wasn't that much complicated and has been already answered. It seems that you do not know how HiLo generator works. Anyway: thank you for sacrificing your time and effort. – Recognize

Gregory, Actually I do know what I am talking about, I have written Batoo JPA that is the %100 JPA Implementation that is currently in its incubation and beats Hibernate in terms of speed - 15 times faster. On the otherhand I might have misunderstood your question and didn't think that using Hibernate with sequences should create any problem at all as I have used Hibernate since 2003 in many projects on many databases. The important thing is you got solution to the question, sorry I missed the answer marked as correct... – Ramp

Sorry, I didn't mean to offend you. Thanks again for your help, question is answered. – Recognize

Answer 4

After digging into hibernate source code and Below configuration goes to Oracle db for the next value after 50 inserts. So make your INST_PK_SEQ increment 50 each time it is called.

Hibernate 5 is used for below strategy

Check also below http://docs.jboss.org/hibernate/orm/5.1/userguide/html_single/Hibernate_User_Guide.html#identifiers-generators-sequence

@Id
@Column(name = "ID")
@GenericGenerator(name = "INST_PK_SEQ", 
strategy = "org.hibernate.id.enhanced.SequenceStyleGenerator",
parameters = {
        @org.hibernate.annotations.Parameter(
                name = "optimizer", value = "pooled-lo"),
        @org.hibernate.annotations.Parameter(
                name = "initial_value", value = "1"),
        @org.hibernate.annotations.Parameter(
                name = "increment_size", value = "50"),
        @org.hibernate.annotations.Parameter(
                name = SequenceStyleGenerator.SEQUENCE_PARAM, value = "INST_PK_SEQ"),
    }
)
@GeneratedValue(strategy = GenerationType.SEQUENCE, generator = "INST_PK_SEQ")
private Long id;

Avner

Sorry, but this is an extremely verbose way of set up something, which can be expressed easily with two parameters for a whole Hibernate and thus for all entities. – Recognize

true but when i try with other ways none of them worked if you have it working can send me how you have configured – Avner

I've updated my answer - now it includes also a working example. Although my comment above is partially wrong: unfortunately, you cannot set neither allocationSize nor initialValue globally for all entities (unless using only one generator, but IMHO it's not very readable). – Recognize

Thanks for the explanation but what you wrote above i have tried and it did not work with hibernate 5.0.7.Final version then i have digged into the source code to be able to achieve this goal and that is the implementation i was able to find in hibernate source code. Configuring might look bad but that is unfortunately hibernates api and i am using hibernate's standard EntityManager Implementation – Avner

Answer 5

Steve Ebersole & other members,
Would you kindly explain the reason for an id with a larger gap(by default 50)? I am using Hibernate 4.2.15 and found the following code in org.hibernate.id.enhanced.OptimizerFactory cass.

if ( lo > maxLo ) {
   lastSourceValue = callback.getNextValue();
   lo = lastSourceValue.eq( 0 ) ? 1 : 0;
   hi = lastSourceValue.copy().multiplyBy( maxLo+1 ); 
}  
value = hi.copy().add( lo++ );

Whenever it hits the inside of the if statement, hi value is getting much larger. So, my id during the testing with the frequent server restart generates the following sequence ids:
1, 2, 3, 4, 19, 250, 251, 252, 400, 550, 750, 751, 752, 850, 1100, 1150.

I know you already said it didn't conflict with the spec, but I believe this will be very unexpected situation for most developers.

Anyone's input will be much helpful.

Jihwan

UPDATE: ne1410s: Thanks for the edit.
cfrick: OK. I will do that. It was my first post here and wasn't sure how to use it.

Now, I understood better why maxLo was used for two purposes: Since the hibernate calls the DB sequence once, keep increase the id in Java level, and saves it to the DB, the Java level id value should consider how much was changed without calling the DB sequence when it calls the sequence next time.

For example, sequence id was 1 at a point and hibernate entered 5, 6, 7, 8, 9 (with allocationSize = 5). Next time, when we get the next sequence number, DB returns 2, but hibernate needs to use 10, 11, 12... So, that is why "hi = lastSourceValue.copy().multiplyBy( maxLo+1 )" is used to get a next id 10 from the 2 returned from the DB sequence. It seems only bothering thing was during the frequent server restart and this was my issue with the larger gap.

So, when we use the SEQUENCE ID, the inserted id in the table will not match with the SEQUENCE number in DB.

Pyromancy

Answer 6

I too faced this issue in Hibernate 5:

@Id
@GeneratedValue(strategy = GenerationType.SEQUENCE, generator = SEQUENCE)
@SequenceGenerator(name = SEQUENCE, sequenceName = SEQUENCE)
private Long titId;

Got a warning like this below:

Found use of deprecated [org.hibernate.id.SequenceHiLoGenerator] sequence-based id generator; use org.hibernate.id.enhanced.SequenceStyleGenerator instead. See Hibernate Domain Model Mapping Guide for details.

Then changed my code to SequenceStyleGenerator:

@Id
@GenericGenerator(name="cmrSeq", strategy = "org.hibernate.id.enhanced.SequenceStyleGenerator",
        parameters = {
                @Parameter(name = "sequence_name", value = "SEQUENCE")}
)
@GeneratedValue(generator = "sequence_name")
private Long titId;

This solved my two issues:

1. The deprecated warning is fixed
2. Now the id is generated as per the oracle sequence.

Rubious

You can use your original mapping as long as you tell Hibernate to use its "new generators" - set hibernate.id.new_generator_mappings to true. And to proactively answer what I can just see as the follow-up... remember that Hibernate is literally 20 years old, and that there are MANY existing deployments and we cannot just change how id generation works by default without breaking all of those existing apps... – Benedikta