Batch inserts with JPA/EJB3

Question

Does JPA/EJB3 framework provide standard way to do batch insert operation...? We use hibernate for persistence framework, So I can fall back to Hibernate Session and use combination session.save()/session.flush() achieve batch insert. But would like to know if EJB3 have a support for this...

Sinegold

Answer 1

Neither JPA nor Hibernate do provide particular support for batch inserts and the idiom for batch inserts with JPA would be the same as with Hibernate:

EntityManager em = ...;
EntityTransaction tx = em.getTransaction();
tx.begin();

for ( int i=0; i<100000; i++ ) {
    Customer customer = new Customer(.....);
    em.persist(customer);
    if ( i % 20 == 0 ) { //20, same as the JDBC batch size
        //flush a batch of inserts and release memory:
        em.flush();
        em.clear();
    }
}

tx.commit();
session.close();

Using Hibernate's proprietary API in this case doesn't provide any advantage IMO.

References

• JPA 1.0 Specification
  • Section 4.10 "Bulk Update and Delete Operations"
• Hibernate Core reference guide
  • Chapter 13. Batch processing

Change

Just for sanity check, your if statement should be i > 0 && i % 20 == 0 else it'll flush (and clear) immediately after the first element was added. – Godhood

Answer 2

With medium records number you can use this way:

em.getTransaction().begin();
for (int i = 1; i <= 100000; i++) {
     Point point = new Point(i, i);
     em.persist(point);
     if ((i % 10000) == 0) {
          em.flush();
          em.clear();
     }
}
em.getTransaction().commit();

But with large records number you should perform this task in multiple transactions:

em.getTransaction().begin();
for (int i = 1; i <= 1000000; i++) {
      Point point = new Point(i, i);
      em.persist(point);
      if ((i % 10000) == 0) {
          em.getTransaction().commit();
          em.clear();          
          em.getTransaction().begin();
      }
}
em.getTransaction().commit();

Ref: JPA Batch Store

Bilodeau

Answer 3

For hibernate specifically, the whole chapter 13 of the core manual explain the methods.

But you are saying that you want the EJB method through Hibernate, so the entity manager documentation also has a chapter on that here. I suggest that you read both (the core and the entity manager).

In EJB, it is simply about using EJB-QL (with some limitations). Hibernate provides more mechanics though if you need more flexibility.

Sandbank

Answer 4

Yes you can rollback to your JPA implementation if you wish in order to have the control you defined.

JPA 1.0 is rich on EL-HQL but light on Criteria API support, however this has been addressed in 2.0.

Session session = (Session) entityManager.getDelegate();
session.setFlushMode(FlushMode.MANUAL);

Saliferous

Answer 5

Pascal

In your example to insert 100000 records, it is done within single transaction, as the commit() is only called at the end.. Does it put a lot pressure towards the database? Furthermore, in case there is rollback, the cost will be too much..

Will the following approach be better?

EntityManager em = ...;
for ( int i=0; i<100000; i++ ) {
   if(!em.getTransaction().isActive()) {
      em.getTransaction().begin();
   }
   Customer customer = new Customer(.....);
   em.persist(customer);
   if ((i+1) % 20 == 0 ) { //20, same as the JDBC batch size
      //flush and commit of inserts and release memory:
      em.getTransaction().commit(); 
      em.clear();
   }
}

session.close();

Nannienanning