Lazy/Eager loading strategies in remoting cases (JPA)

Question

I'm running into LazyLoading exceptions like the most people who try remoting with an ORM. In most cases switching to eager fetching solves the problem (Lazy Loading / Non atomic queries / Thread safety / n+1 problem ...). But eager fetching has also disadvantages if you are dealing with a really big object graph.

Loading the whole object graph isn't needed in the most use-cases. It feels bad to load more data then needed (or load them from the db and extract the needed subset).

So what alternative ways are there to solve this kind of problem (at runtime)?
I've seen:

• Inject a data access dependency into domain object and let the object decide either to load lazy or eager: Feels bad! The domain layer should be independent from any service. Domain injection is also an expensive operation. The domain should be data access ignorant and should be used with or without data access.
• Fetch everything lazy except of use-cases which require more data: Seems better for performance but this way forces many client=>server / database roundtrips. The initialisation of the lazy fields can also suffer pain (tried with JPA). This way doesn't feel generic and is subject of the same lazy restrictions mentioned above.
• Encapsulate persistence in Lazy class: More complexity, no best practice for interoperation with ORM. Bloating services layer (so much "hand written" code feels bad).
• Use full projections for every use-case: We'll end up in SQL and drop the benefit of an ORM.
• A DTO / Virtual Proxy layer enforces more complexity and makes code harder to maintain (Wormhole antipattern >> Bloat).

I thought a lot about another way. Maybe generic projection white./black listning is a solution.

Idea (blacklist): Define an classname list with the boundaries for a fetching operation. If a property matches and it's lazy, remove the lazy (CGLIB) proxy and fill the value with null. Else, simple prevent from fetching (and leave value at null). So we can set clear boundaries in our DAOs.

Example: ProductDao.findByName("Soap",Boundaries.BLACKLIST,"Category, Discount") the two last parameters can also been bound into a Boundaries object.

Idea (whitelist): Like blacklist, but you must declare properties with should be loaded in a whitelist.

What do you think about such a solution? (Possible problems, restrictions, advantages ...) How should I write this in java? Maybe via AOP to match DAO methods (because I'm able to modifiy cglib proxy behaviour there)?

Delsiedelsman

What kind of architecture do you have? Are you using GWT, for instance? – Sorbitol

JAX-WS Webservices via reference implementation (Metro) – Delsiedelsman

Answer 1

1. You can get rid of all collections whatsoever and use NamedQueries instead. We used this approach in one project (EJB + Swing), and it worked pretty well - thus you determine exact data to be fetched. NamedQueries are normal queries, imagine them as PreparedStatement-s. The idea is not to create/retreive/update/delete single objects with queries. The idea is that you fetch your Collections with queries. For example, instead of mapping a @ManyToMany List, define a NamedQuery that fetches that list. Thus you can fetch the collection data separately, and only whenever you need it, not automatically.

2. Use a custom Proxy (using CGLIB) for transferred objects - whenever a collection is referenced (via its getter), attempt retreival, and catch any LazyInitializationException and make a call to the server tier for the data requested.

3. Just as the previous one, but make proxies only of the collections, in the way Hibernate proxies them when lazy initialization is needed.

4. Also, take a look at the Value List Handler pattern - might be useful.

(You can also use hibernate.max_fetch_depth (if using Hibernate) with a combination of the above, if it is suitable for your case.)

Mra

Named Queries end up in code duplication for CRUD operations if they are applied to every domain class. I'm using Generic JPA DAOs wired and created by spring. It's also very important to avoid the hibernate dependency. Decorator/Proxy via AOP seems nice. But your approach isn't decoupled from the data layer (domain/service mix | domain/data access mix) cause the domain getter should make server calls. I try to avoid injection into the model (logic for calls etc.) like described in my question. – Delsiedelsman

Well, Hibernate for example uses the same idea to fetch lazy collections when the session is still there - it makes a proxy of the collection that do contain logic. So it is not a big deal to do it. NamedQueries are not duplication of the CURD operation, but are instead of the @*ToMany annotations (and the related stuff) - a little more verbose, though. – Mra

P.S. Why is it important to avoid Hibernate dependency? If you are using Hibernate as persistence provider, you will eventually end up using its annotations in cases JPA doesn't have the required option. :) – Mra

We use EclipseLink very often. JPA fulfils all our needs and we don't want to use Hibernate annotations. Why are NamedQueries not CRUD duplicate? See: bit.ly/7WEdaI Professor.findByPrimaryKey which is the same as GenericDao<Professor>.read(PK)! How are NamedQueries different from general JPA Queries? Are there still lazy proxies for references which aren't affected by the query? – Delsiedelsman

And, JPA fulfills your needs NOW. From my experience one almost always needs to revert to native JPA provider tricks at some point. For example, how do you handle the "delete-orphan" case with JPA? – Mra

You have to handle the synchronisation of detached objects by yourself if you don't use features like private ownership or delete-orphan. The handling of such cases (or QueryByExample which isn't part of the JPA spec) are handled generic written in our DAO layer. JAX-WS tries to serialize every entity. So it fails on every lazy field (out of the persistence context). Means: Named Queries are useful to decide which data I want to load. But it doesn't solve the problem in any case. Catching any LazyInitializationException with proxies/aop should do the trick. I'll try it. – Delsiedelsman

OK. :) Just to add, that the case in our project was the same (except it was RMI and not JAX-WS), and because we used named queries there was no place a LazyInitializationException could occur - there were no colelctions in the entities. – Mra

Is there a gerneric way to handle named queries and annotations in a generic DAO structure? – Delsiedelsman

What do you mean? We defined the @NamedQueries as annotations on each entity. The DAO just delegates to the EntityManager. – Mra

One benefit of the DAO is, that you can change the behaviour of the persistence strategy independent from the model (or only as special as needed). With NamedQueries you'll bind your persistence strategy to the model. Are NamedQueries with a generic behaviour for every entity (in the DAO?) also possible? Aren't NamedQueries for every association against the general JPA design? – Delsiedelsman

OpenJPA seems to have a feature called "FetchPlan" wich can define "FetchGroups" at runtime. I like to see this in general JPA. I'll try to write such persistence logic in my dao. bit.ly/4xjg1X – Delsiedelsman

Answer 2

Nowadays (2013), it is possible to keep lazy-loading if you remote your service with GraniteDS.

That should properly serialize your JPA or Hibernate entities by not initializing lazy relations and keeping the lazy-state for the client. If you access those relations on the client, it will fetch them transparently in the background.

Moreover, GraniteDS seems to be able to do inverse lazy loading which means that when you send modified objects back to the server, it will not send unchanged entities back thus making the necessary server communication very efficient.

I am not a GraniteDS expert (yet) but it seems to be able to integrate with JEE6 and Spring service layers and works with all of the most important JPA providers.

Naturally, you will need to hide the GraniteDS based remoting behind a service interface for maximizing transparent behaviour but that can be easily be done if the client also uses Spring (so you inject the service according to the needs of the environment).

Coastal

Answer 3

Although it takes a bit of work and for JAX-WS/JAXB requires recent versions of those libraries, this is a very elegant solution: create a marshaller that can test for whether objects / collections have been initialized.

As described here: https://forum.hibernate.org/viewtopic.php?f=1&t=998896

Malayalam