+ Start a Discussion

Java SOAP Compression - With AXIS 1.2 - An Epic Tale of WOAH....

(Ok, the title could use work, but who am I, Steven King?)

In the process of preparing to migrate a bunch of integrations away from .NET (my client is going to be switching to Firefox with or shortly after the new release) I began work in earnest converting things to Java, and Axis. Most of the simple stuff came across fairly easily. Printing is another story entirely, but performance became an early problem.

Async calls in Axis seem to be _very_ undocumented. In fact I couldn't find any examples at all, and only a small directory of source code in the Axis source to guide me. After putting Async calls on the back burner I turned to SOAP compression. Before I get into my tale let me set some background. I hate modifying 'stock' code, and would much rather extend and override it where I can. The SFDC technote, while I'm soure it would work, did exactly this. (Seeing as I can't control _every_ machine that would get these applets deployed I can't depend on a modified Axis distribution.)

Off I went looking for solutions to the SOAP compression problem. Why neither .NET or Axis allow for (at least) receiving a gzip-encoded response is beyond me. Nonetheless Google is your friend and I quickly found this library for Axis SOAP compression... And there was much rejoicing.

That was until I actually tried it. Problem is that library makes a call to close the outbound socket. While normally not a problem, the Java SSL sockets really don't like this. In fact it pukes up a very nice exception for you. I even went as far as to disassemble the library and comment out those pieces of code. That didn't get far either as it appeared to just hang at that point waiting to read from the server.

From time to time you find a problem that just kicks your ass. The human spirit triumphs and you get determined to kick it's ass back into submission.

Two days later, holding my sorry and very disfigured ass after getting it handed to me, repeatedly, by SSL sockets in Java, I had pretty much rewritten the Axis HTTPSender routiene. I'd like to say it was all working quite well now, but I can't, in fact it had gotten about nowhere. Invariably what would work with SSL would fail without, or vice versa. Time to try another angle.

One of the stock Axis transports uses the Apache Commons HTTPClient library. Oh joy of joys, perhaps it knows how to do gzip encoding? Nope. Bzzt, sorry Hans, wrong answer...

But there just might be a way to do it. Another 8 hours later I find this interesting message howing how the HTTPClient PostMethod can be abstracted and overridden to allow for receipt of gzip encoded data. Not bad. Halfway there.

Copy and paste that into a class in Eclipse, and modify the one line in org.apache.axis.transport.CommonsHTTPSender.java, making it into my own class int he process, and vola it works!

The moral of the story: What a $%#@ PITA. And I still haven't even gotten to async calls yet...

If anyone is interested in my .java files I used to get here, or the ones that are FUBAR, let me know and I'll be happy to forward them over. My email address should be visible in my profile.

I'm going to wait a few days and give another crack at outbound compression and modifiying what I've got working so far... If anyone wants to chip in, let me know.
When the source to the Sforce Data Loader gets posted, you'll find a sender class based on the axis CommonsHTTPSender class, that supports bidirectional compression. This is a stand-alone class, you alter the client-config.wsdd class to have axis use it instead of its stock one, no need to rebuild the axis jar.
Thats essentially what I did - but I stoped at unidirectional.

(P.S. Feel free to move this to the Java forum as I misposted it here)

I couldn't see your email address in you're profile. If you could change that or send me the file directly that would be good. (Others can email me for the code afterwards)

I've just followed SForce's tech tip to include gzip compression, although I think I'm about to run down the road you just took as weird connection timeout exceptions are popping up.

I'm holding off sending my code out: See Simon's post just below here and you'll see why -- SFDC already wrote a class to do just this -- bidirectionally.
Saw that the CommonsHTTPSender in the dataloader jar has the pkg common.api.soap - earlier it was org.apache.axis.client.
This class is packaged in the dataloader jar but I want just this class for the soap compression and do not need the other dataloader classes. I guess I will need the client-config.wsdd also. What is the recommended way to use these 2 files in a packaged solution i.e. apart from including the whole dataloader jar?
just grab the source (its BSD licensed for this very reason) and compile it into you app.
Thanks Simon. I was looking at the commonshttpsender code in dataloader and saw that it compresses based on the sfdc.sendGzip property being set in the MessageContext. How do I actually set this property in the messagecontext? Is that by doing SoapBindingStub.setProperty(...)?
Yes, or you can subclass your service locator class to set that property on every call, by overriding the createCall method.