How to merge common parts of WSDL and XSD from different services?

Question

I have to interact with a set of web-services that each come with their own WSDL and XSD. The XSD are sometimes merged in a single file sometimes spread along multiple files (20-30). However, from experience I know that most of the message structure and data share a large common subset, perhaps only 20% are different amongst the different transactions.

Unfortunately I have no control over the server parts or the declaration of the services so getting them to fix it is out of the question. A first version of the client generated each services separately and then used them as individual facades to form a coherent higher level service as an adapter for another system.

I used CXF with the default JAXB binding and imposed different generated packages for each services. I did this because some most services use a common data model but not all use the same version or customization so I have conflicts and thus opted for the brute force so I can get the system done.

However, this causes the memory requirements of the adapter to go through the roof as each services load their context. Right now I have upwards 500M of memory utilized just for the adapter that houses the service clients even before I start sending requests and processing responses. Although I can run the system without problems using current situation this create constraints that jeopardize the deployment of the solution; my client would like to reduce this dramatically (60% or more) so that this system can be installed along side others without requiring hardware upgrades.

Question is follows : Is there a tool or technique that would allow me to put the common parts of each transactions together such that they can be generated once and referenced where needed ?

I am not bound to CXF or JAXB other than the time required to re-factor the system towards a different framework or data bindings.

Thank you in advance for your help.

--- EDIT ---

Thank you Blaise. This points to a feature of JAXB that would be useful : episodes. Unfortunately I still need to extract the common base part of the different services. So now what I need is a means to extract this common parts through a structural diff, that is a diff tool that would be aware of the structure and type hierarchy the XSD describes so that proper references be put in place to connect the common sections with the specialized parts.

Nodical

You may be interested in following this related feature in EclipseLink JAXB (MOXy): bugs.eclipse.org/340997 – Tripersonal

Are they using a common namespace for the shared parts? If not, you're SOL because the various parts simply don't understand that they're the same. – Aeriell

How much does 1Gb of memory cost? How much does your time cost? Point this out to the client. – Pinole

@donal : sorry for the delay, must have missed the comment notification. Yes, fortunately they all share the same namespace so it is, theoretically, possible to exploit similarities, at least regarding namespaces. – Nodical

@Pinole : true enough, 1G is not that much, however, other security requirements had us run the whole thing in the same windows session as the interactive user. Still, not so bad until you get to deploy on a Remote Desktop server at which point having sessions jump from 150m to 1G+, the implications on server load wre just not practical. That said, We did separate the part that was bound to the confines of user interactive sessions from the rest, now the "client" side is a more reasonable 15m. Still all this is moot as the boss said "unacceptable!" and will not hear any other "excuses". – Nodical

Answer 1

If you want to trim down a little, an alternative marshalling technology (in any framework) might do the trick - drop JAXB and try JiBX, which was added to the latest CXF release, or maybe just StAX.

Since you're looking to do something a little more custom than the conventional JAX-Ws/JAXB style services, you may want to consider Spring-WS.

Spring-WS gives you control over all aspects of the web services stack. It can route messages in different ways (payload, XPath expressions, etc), and you can use any marshalling/serialization technology you want (Jibx, jDOM, SAX, etc)

Here is a table that illustrates the options: http://static.springsource.org/spring-ws/sites/2.0/reference/html/server.html#d4e1062

If you really want to get fancy, you can take one of the lower level APIs, start marshalling the message and once you hit critical mass for one of your common areas, start a JAXB marshall right on the spot.

The ability to route messages to different 'endpoints' (in Spring-WS) terms, means you can also do things like "accept any message" on this one interface (that looks like DOM/SAX/etc) and then have one big marshalling operation there.

The key thing Spring-WS will buy you here is to break out of the JAX-WS mold, do play a little up front game, and then you can always marshall back to JAXB later, whether it be in interceptors, your app, etc. In theor you can the same with JAXB DOM Source, but it's my opinion that the Spring-WS stack gives you the finest grained control for special situations like you have here.

Fuddyduddy

I have tried with JiBX, XMLBeans and SDO as alternative bindings for JAXB. some offered some gains but they all remained way too high to be practical. The best I could hope for from my test was a memory gain of approximately 20%, not too shabby but I really am aiming for 80%+ ... – Nodical

Spring-WS however offered more, so I ended up using Spring-WS as a framework for pretty much the reasons you stated. Thus far the only viable solution we have evaluated was to forego the XSD completely and generate the XML through XML Transformation. Best answer yet ! – Nodical

Answer 2

The best trick is to serve a static wsdl. Just open the wsdl, save it, upload in the server and indicate to the client to point to the static one instead of the dynamic-self generated.

Quicktempered

hmmm... The generation I am having trouble with is the code generation from the WSDL and not the other way around. Actually the WSDL I have are static provided by the people managing the server side. Ha, and, as I stated, I do not have access to the server. Perhaps I was not clear enough in the question but I'm quite certain to have covered all this already. – Nodical