Restlet streaming with Thread.sleep()

Question

This example is based on an example from the book Restlet in Action.

If I try

public class StreamResource extends ServerResource
{
    @Get
    public Representation getStream() throws ResourceException, IOException
    {
        Representation representation = new WriterRepresentation(MediaType.TEXT_PLAIN)
        {
            @Override
            public void write(Writer writer) throws IOException
            {
                String json = "{\"foo\" : \"bar\"}";
                while (true)
                {
                    writer.write(json);
                }
            }
        };

        return representation;
    }
}

it works and it continuously sends the json string to the client.

If I introduce a delay in the while loop like this

String json = "{\"foo\" : \"bar\"}\r\n";

while (true)
{
    writer.write(json);
    try
    {
        Thread.sleep(250);
    }
    catch (InterruptedException e)
    {} 
}

I was hoping that the client would get data 4 times in a second BUT nothing seems to get to the client.

Can anyone explain why the introduction of Thread.sleep() does that? What is a good way to introduce delay in streaming data to the client?

Slouch

It is really not a good idea to delay the streaming as this potentially keeps the HTTP connection open for that while. Instead of doing that, you may delay on the client side before getting next json text from the server. – Spiry

It's probably not but I'm trying to avoid manual polling by client. – Slouch

You are using while(true) in your code, which iterates forever until your application crashes. – Spiry

@Indoknight I'm trying to put in some conditions so it will terminate the loop when the client disconnects. – Slouch

Did you check this? rfc2616.wordpress.com/2010/11/16/… – Spiry

@Indoknight yes, and the basic streaming works.. it's just that when I try to introduce some delay in the thread that's writing the stream, I run into problems – Slouch

I'm not an expert on this area, but trying to throw some ideas. Can you try to gradually increase the time delay from milliseconds? – Spiry

yes and I really appreciate your suggestions.. I did try 10 milliseconds and it seems to work but if I get up to 50ms, it becomes erratic and it seems to be blocked for few seconds and then all of a sudden it streams a few seconds worth of data.. at around 100ms it just stops responding. – Slouch

just in case and to - maybe - save you a lot of time: don't try to stream data like this (with Thread#sleep(x)) with enabled compression (i.e. having set getEncoderService().setEnabled(true)) - this might lead to the situation described here. – Smirch

Answer 1

You should try with the Jetty connector instead of the internal Restlet connector. This connector isn't ready for production even though we are working on fixing it.

You can also try the Simple extension which has less dependent JARs than the Jetty extension.

Becalmed

Answer 2

You can try to flush the buffer, like this:

String json = "{\"foo\" : \"bar\"}\r\n";

while (true)
{
    writer.write(json);
    writer.flush();  // flush the buffer.
    try
    {
        Thread.sleep(250);
    }
    catch (InterruptedException e)
    {} 
}

Without writer.flush(), the writer waits to fill the internal buffer before writing the socket. Thread.sleep(250) reduces the output produced at each second, so that far more time is required to fill the buffer.

Cyte

Thanks for your response. I did try calling flush() when I was tinkering around but it didn't do a difference. They I looked up [Representation#write())[restlet.org/learn/javadocs/2.2/jse/api/org/restlet/… that WriterRepresentation has to override and it states that the class implementing this method shouldn't flush or close the given Writer after writing to it as this will be handled by the Restlet connectors automatically and figured that's the reason it didn't work. – Slouch

Mmmm... I'm still not sure. The connector can flush and close the writer only when the WriterRepresentation.write() method returns. And your infinite loop prevent this. Thus writer.flush() should work. – Cyte

Could you try with the Jetty connector instead of the internal Restlet connector? Does that make a difference? – Becalmed

@JeromeLouvel on the other hand, if I try just using jetty (no restlet) like this, it works as intended. I have a lot of restlet codebase so I'm trying to get it to work with restlet's ServerResource. Any help would be very much appreciated! Thanks – Slouch

@JeromeLouvel I take back what I had previously mentioned. It works with my ServletResource using Jetty connector. Is it a limitation of restlet connector? Can I work around it without having to have jetty dependency ? I'm so glad I was able to see it work today after being stuck on it for a week. Thanks. If you can convert your above comment to an answer to this question, I'll give you most of the bounty. – Slouch