How to correctly limit bandwith usage with Java netty?

Question

For a download client that I developed in Java using the netty nio lib I also implemented a bandwith limitation feature. Technically I do so via a GlobalTrafficShapingHandler object. Based on this class' JavaDoc I initialize the nio client pipeline as follows:

...
trafficHandler = new GlobalTrafficShapingHandler(
        new HashedWheelTimer(), 0, 0, 1000);
execHandler = new ExecutionHandler(
        new OrderedMemoryAwareThreadPoolExecutor(20, 0, 0));
...

public ChannelPipeline getPipeline() throws Exception
{
    // create default pipeline
    ChannelPipeline pipeline = pipeline(); 

    // traffic shaping handler
    pipeline.addLast("global-traffic-shaping", trafficHandler);

    // SSL support
    if(useSSL)
    {
        SSLEngine sslEngine = createSSLEngine();
        pipeline.addLast("ssl", new SslHandler(sslEngine));
    }

    // memory executor
    pipeline.addLast("memory-executor", execHandler);

    // business logic
    pipeline.addLast("data-processing", 
        new NettyNioClientHandler(
            nettyNioClient, localer, logger, ncMgr, username, useSSL));

    return pipeline;
}

And during runtime I then set the max. download speed via

public void setDlSpeedLimit(long limit)
{
    if(limit < 0)
        return;

    trafficHandler.configure(0, limit * 1000L);
}

Ok, so basically the netty nio functionality works fine and fast. When I set the max. download speed in the application, I can also see that the bandwidth usage is indeed capped at a max. level. I monitor the bandwith usage via

trafficHandler.getTrafficCounter().getLastReadThroughput();

However, unfortunately the max. speed I monitor is not to what I have set it before, not even close. For example, I originally (w/o limit) have a download speed of about 2000 kb/s, then I set the limit to 300 kb/s as described above, but the real download speed is then varying from 700-900 kb/s.

So my problem in this case is: I can see that the traffic shaper is doing something, but not as I wanted it to. Do I miss something here, any step in the pipeline initialization for example?

Thanks in advance for your help!

Ragi

Answer 1

Ok, so it seems there are not other ideas.
The only thing that helped me a bit was to increase the timing counter to 5-10 sec.

Cheers!

Ragi

Answer 2

Your getPipeline() is instantiating GlobalTrafficShapingHandler every time a new pipeline is requested. You should only instantiate one and reuse it in all pipelines instead. This is also true for other classes, such as HashedWheelTimer.

Netty has quite a few classes that must be shared among pipelines. Usually the documentation clearly mentions this. You should examine each classes' javadoc and make sure you are following them.

Veery

You are of course right, in reality I did as you suggested in my code. The GlobalTrafficShapingHandler, HashedWheelTimer, and ExecutionHandler is created before in the class constructor. Sorry if my original post has confused you in that regard. I have updated it accordingly. – Ragi