Long polling PHP returns 2 results instead of one
Asked Answered
A

12

13

I'm trying to create a Posting System just like Facebook. So I did a little bit research about how Facebook does it, Facebook uses long polling، So I searched around on how to implement it, I implement it. And I finally finished it, I opened both Firefox and Chrome to test it out. After 2 or 3 posts it worked, but then it will duplicate the results. As you can see below:

Duplicate results

It's the first post by the way.

And here is my network tab, During that process: It makes 3 requests instead of two

It makes 3 requests instead of one.

And finally here is my code:

init.js that contains all of my JavaScript code

function getNewPosts(timestamp) {
  var t;
  $.ajax({
    url: 'stream.php',
    data: 'timestamp=' + timestamp,
    dataType: 'JSON',
})
  .done(function(data) {
    clearInterval( t );
    // If there was results or no results
    // In both cases we start another AJAX request for long polling after 1 second
    if (data.message_content == 'results' || data.message_content == 'no-results') {
        t = setTimeout( function() {
            getNewPosts(data.timestamp);
        }, 1000);
        // If there was results we will append it to the post div
        if (data.message_content ==  'results') {
            // Loop through each post and output it to the screen
            $.each(data.posts, function(index, val) {
                $("<div class='post'>" + val.post_content + "<div class='post_datePosted'>"+ val.posted_date +"</div> <br>" + "</div>").prependTo('.posts');
            });
        }
    }
})
}

$(document).ready(function(){

    // Start the autosize function
    $('textarea').autosize();

    // Create an AJAX request to the server for the first time to get the posts
    $.ajax({
        async: false,
        url: 'stream.php?full_page_reload=1',
        type: 'GET',
        dataType: 'JSON',
    })
    .done(function(data) {
        // Assign the this variable to the server timestamp
        // that was given by the PHP script
        serverTimestamp = data.timestamp;
        $.each(data.posts, function(index, val) {
            $("<div class='post'>" + val.post_content + "<div class='post_datePosted'>"+ val.posted_date +"</div>" + "</div>").prependTo('.posts');
        });
    })
    .fail(function() {
        alert('There was an error!');
    })
    // When the form is submitted
    $('#post_form').on('submit', function(event) {
        $.ajax({
            url: 'ajax/post.php',
            type: 'POST',
            dataType: 'JSON',
            data: $('#post_form').serialize()
        })
        .done(function(data) {
            // Reset the form values
            $('#post_form')[0].reset();
        })
        .fail(function() {
            // When there was an error
            alert('An error occured');
        })
        // Prevent the default action
        event.preventDefault();
    });
    // Start the actual long polling when DOM is ready
    getNewPosts(serverTimestamp);
});

And my stream.php

<?php
header('Content-type: application/json');
// If it was a full page reload
$lastId = isset($_GET['lastId']) && !empty($_GET['lastId']) ? $_GET['lastId'] : 0;
if (isset($_GET['full_page_reload']) && $_GET['full_page_reload'] == 1) {
    $first_ajax_call = (int)$_GET['full_page_reload'];

    // Create a database connection
    $pdo = new PDO('mysql:host=localhost;dbname=test', 'akar', 'raparen');
    $sql = "SELECT * FROM `posts`";
    $stmt = $pdo->prepare($sql);
    $stmt->execute();
    $posts = $stmt->fetchAll(PDO::FETCH_ASSOC);
    // Output the timestamp since its full page reload
    echo json_encode(array(
        'fullPageReload' => 'true',
        'timestamp' => time(),
        'posts' => $posts
        ));
} else if (isset($_GET['timestamp'])) {
    // The wasted time
    $time_wasted = 0;
    // Database connection
    $pdo = new PDO('mysql:host=localhost;dbname=test', 'akar', 'raparen');
    $timestamp = $_GET['timestamp'];
    // Format the timestamp to SQL format
    $curr_time = date('Y-m-d H:i:s', $timestamp);
    $sql = "SELECT * FROM `posts` WHERE posted_date >= :curr_time";
    $stmt = $pdo->prepare($sql);
    $stmt->bindValue(':curr_time', $curr_time);
    $stmt->execute();
    // Fetch the results as an Associative array
    $posts = $stmt->fetchAll(PDO::FETCH_ASSOC);
    // If there wasn't any results
    if ( $stmt->rowCount() <= 0 ) {
        // Create the main loop
        while ($stmt->rowCount() <= 0) {
            // If there is still no results or new posts
            if ($stmt->rowCount() <= 0) {
                // If we waited 60 seconds and still no results
                if ($time_wasted >= 60) {
                    die(json_encode(array(
                        'message_type' => 'error',
                        'message_content' => 'no-results',
                        'timestamp' => time()
                        )));
                }
                // Helps the server a little bit
                sleep(1);
                $sql = "SELECT * FROM `posts` WHERE posted_date >= :curr_time";
                $stmt = $pdo->prepare($sql);
                $stmt->bindParam(':curr_time', $curr_time);
                $stmt->execute();
                $posts = $stmt->fetchAll(PDO::FETCH_ASSOC);
                // Increment the time_wasted variable by one
                $time_wasted += 1;
            }
        }
    }
    // If there was results then we output it.
    if ($stmt->rowCount() > 0) {
        die( json_encode( array(
            'message_content' => 'results',
            'timestamp' => time(),
            'posts' => $posts,
            )));
        exit();
    }
}

And here is my ajax/post.php:

<?php
if ( isset($_POST['post_content']) ) {
    $post_content = strip_tags(trim($_POST['post_content']));
    if ( empty($post_content) ) {

        /* If the user doesn't enter anything */
        echo json_encode(array(
            'message_type' => 'error',
            'message_content' => 'It seems like your post is empty'
            ));
    } else {
        $pdo = new PDO('mysql:host=localhost;dbname=test', 'akar', 'raparen');
        $sql = "INSERT INTO `posts` (`post_id`, `post_content`, `posted_date`) VALUES (NULL, :post_content, NOW());";
        $stmt = $pdo->prepare($sql);
        $stmt->bindValue(':post_content', $post_content);
        $stmt->execute();
        echo json_encode(array(
            'message_type' => 'message',
            'message_content' => 'Your post has been posted successfully.'
            ));
    }
}

If you don't understand it just ask me. I know it's dirty code and I repeated myself a lot. I did that for testing, so it doesn't really matter.

Thanks!

Aromaticity answered 7/10, 2014 at 9:45 Comment(14)
If it duplicates results, you should also post the source of ajax/post.phpGasman
It's problem with the stream.php or init.js I posted it anyway.Aromaticity
Long polling must infinite loop to call it "Long Polling".Aromaticity
You are using "serverTimestamp" before it is defined, since javascript is asynchronous you should place the getNewPosts inside of the .done()-function of your "first time to get the post request"Agapanthus
Yeah it does it makes your code less error-prone, I'm not giving you an answer here I'm only commenting.Agapanthus
How about replacing your query condition posted_date >= :curr_time with posted_date >= :curr_time. Anyway I don't think it's a good way to implement. What if the time of the web server and the time of database server are different?Saxena
:curr_time is the current server time that I got when the page is loaded. And I don't think that will happen.Aromaticity
Does your db show duplicate posts as well?Gasman
Nope, When I refresh the page, It doesn't duplicate.Aromaticity
What is the content of the requests in your network tab, are there duplicates sent by the server?Imponderabilia
Yep there are, Instead of finishing the first request and start another one, It finishes the first request, creates another one with the first details and start another one. So It's 3 requests instead of 2. I hope you understand.Aromaticity
You may be losing the scope of your interval ( t ) in the top is defined inside of a function, getNewPosts, if so you will see your intervals will start to stack up and send multiple requests every second. first one, then 2 then 3 because its not clearing them out.Midas
I guess you are working with multiple declarations of an id(#post_form). You have also a lot of JavaScript erros(undefined t, clearInterval and you are using a timeout, ...). Did you check your console for errors and can you show me the html part?Cp
I had similar problem. Try unseting event listener before you add a new one, just like : $('#post_form').off('submit').on(..)Australorp
F
1

Frankly, I don't see why you bother with this kind of optimization, unless you plan to handle thousands of messages. You could as well fetch the whole lot each time the page gets refreshed.

Hammering the server with a request from each client every second will generate a huge lot of traffic anyway, so optimizations should start with defining a more reasonable polling period or a smarter, adaptive refresh mechanism, IMHO.

Now if you really want to go for it, you will have to do a proper synchronization. If you mess up the timestamps, you can skip a message that was added by someone else just as another client triggered an auto-refresh, or get the said message twice.

All your timeout handling is simply unnecessary. A server request through AJAX will produce an error event if something goes wrong, which will mean either the connection or your server went down or your PHP-side code threw some tantrum and needs a fix.

For an idea of the application structure:

  • an update request will pass a timestamp to PHP, asking to retrieve all posts newer than the said timestamp. The initial value will be 1/1/1970 or whatever, so that the initial fetch retrieves all existing posts. A new timestamp will be included in the response, so that incremental requests will skip already fetched data.
  • JavaScript will generate such requests periodically (I would rather set the period to 30 seconds or so, to avoid excessive server load - assuming your average user can handle the frustration of waiting that long for the next batch of pseudo-tweets)
  • submitting a new post will simply add it to the DB, but since all is done server-side, you won't need to bother with race conditions.

All your "time_wasted" and "cur_time" code should go to the bin. The only time reference needed is the date of the last read request from this particular client.
All you need server-side (in your "stream" PHP file) is a DB request to fetch posts newer than the client-provided timestamp, that will return a (possibly empty) list of posts and the updated value of the same timestamp.

Frankly, instead of these potentially confusing timestamps, you could as well use the unique identifier of the last post fetched (using 0 or any conventional value for the initial request).

Floodlight answered 1/11, 2014 at 4:47 Comment(1)
I did it today. I have a code that checks the database every 15 seconds, instead of long polling. Thanks!Aromaticity
C
1

You set a timeout like:

setTimeout()

but you are using

clearInterval()

to clear it, use clearTimeout() instead.

Cabob answered 14/10, 2014 at 11:8 Comment(0)
E
1

I know this does not answer your question exactly, but what you're doing is not gonna work anyway - long polling with PHP will crash your server when there'll be at least some more users. You use sleep, so PHP process is "hanging". PHP worker count (both for Apache, nginx or any server with PHP) is limited. As soon as you'll reach that count, new connections will be rejected. PHP is intended to give responses quickly.

For this type of solution I would suggest to use some intermediate software that's designed for it. For example, take a look at Socket.IO.

It's written in Javascript and is for both client side (JS library) and server side (Node.js). Your Node.js server can take events as they happen from PHP using REST API, queues (like ZeroMQ, RabbitMQ etc) or any other transport, like socket.IO client itself. This way you don't poll your database in PHP, you just pass that new post was added to the Node.js server, which passes this information to your client-side JS code.

$pdo->prepare('INSERT INTO ..')->execute();
$dispatcher->dispatch('new_post', new PostEvent(array('id' => 123, 'text' => 'Post text')));

Long-polling is only one of supported Socket.IO protocols, and is by far not the most efficient one.

If you want to avoid Node.js, you can try ReactPHP with Ratchet, using WebSockets on client side. This works as single php process (ran from command line), thus not apache-way.

Erratic answered 15/10, 2014 at 18:36 Comment(1)
this is maybe not the direct answer but in the long run definitely it is the correct way. Note: yes i am a PHP developer. and yes i have used both.Elegant
F
1

Frankly, I don't see why you bother with this kind of optimization, unless you plan to handle thousands of messages. You could as well fetch the whole lot each time the page gets refreshed.

Hammering the server with a request from each client every second will generate a huge lot of traffic anyway, so optimizations should start with defining a more reasonable polling period or a smarter, adaptive refresh mechanism, IMHO.

Now if you really want to go for it, you will have to do a proper synchronization. If you mess up the timestamps, you can skip a message that was added by someone else just as another client triggered an auto-refresh, or get the said message twice.

All your timeout handling is simply unnecessary. A server request through AJAX will produce an error event if something goes wrong, which will mean either the connection or your server went down or your PHP-side code threw some tantrum and needs a fix.

For an idea of the application structure:

  • an update request will pass a timestamp to PHP, asking to retrieve all posts newer than the said timestamp. The initial value will be 1/1/1970 or whatever, so that the initial fetch retrieves all existing posts. A new timestamp will be included in the response, so that incremental requests will skip already fetched data.
  • JavaScript will generate such requests periodically (I would rather set the period to 30 seconds or so, to avoid excessive server load - assuming your average user can handle the frustration of waiting that long for the next batch of pseudo-tweets)
  • submitting a new post will simply add it to the DB, but since all is done server-side, you won't need to bother with race conditions.

All your "time_wasted" and "cur_time" code should go to the bin. The only time reference needed is the date of the last read request from this particular client.
All you need server-side (in your "stream" PHP file) is a DB request to fetch posts newer than the client-provided timestamp, that will return a (possibly empty) list of posts and the updated value of the same timestamp.

Frankly, instead of these potentially confusing timestamps, you could as well use the unique identifier of the last post fetched (using 0 or any conventional value for the initial request).

Floodlight answered 1/11, 2014 at 4:47 Comment(1)
I did it today. I have a code that checks the database every 15 seconds, instead of long polling. Thanks!Aromaticity
A
1

you can use this:

$pdo->prepare('INSERT INTO ..')->execute();
$dispatcher->dispatch('new_post', new PostEvent(array('id' => 123, 'text' => 'Post text')));
Allheal answered 22/11, 2014 at 8:13 Comment(0)
G
0

I think there is unecessary code in here. All thet you need is. Define 2 parts. 1- Is your form. 2- Is your message viewer.

Ok so first time the form is loaded, go to database retrive informations (it could be as JSON) and populate your viwer. To do that, inside ajax done, Convert php JSON to an array, create a loop . For each element use append jquery to add each post. http://api.jquery.com/append/

DO the same for the evenemnt on click (submit). You must to clear htm content before to populate your new posts.

Griggs answered 9/10, 2014 at 19:2 Comment(0)
N
0

As you have indicated in you comments, there is allot of redundant code, which makes diagnosing the problem difficult. It would be worth tidying up, just so that other people reading the code are in a better position to diagnose the problem.

Having reviewed the code, this is what I can see happening.

  1. Dom Ready functions starts the ajax request for full load
  2. Dom Ready functions starts the getNewPosts() with default server time
  3. Full ajax load returns
  4. getNewPosts() returns

You can verify this order by adding console.log() commands to the various functions. The exact order may vary depending on how quickly the server responds. However, the underlying problem is that the serverTimestamp value is not set when the step 2 is started.

The resolution is easy enough, the serverTimestamp variable needs to be correctly set. To do this move the getNewPosts() function call to the .done() handler for the full load ajax request. At this point the server has returned an initial time stamp value that can be used for further polling.

// Create an AJAX request to the server for the first time to get the posts
$.ajax({
    async: false,
    url: 'stream.php?full_page_reload=1',
    type: 'GET',
    dataType: 'JSON',
})
.done(function(data) {
    // Assign the this variable to the server timestamp
    // that was given by the PHP script
    serverTimestamp = data.timestamp;
    $.each(data.posts, function(index, val) {
        $("<div class='post'>" + val.post_content + "<div class='post_datePosted'>"+ val.posted_date +"</div>" + "</div>").prependTo('.posts');
    });

    // Start the long poll loop
    getNewPosts(serverTimestamp);
})
Noma answered 14/10, 2014 at 11:4 Comment(4)
Hmm... I did that too, But it still duplicates.Aromaticity
@Aromaticity Try placing the serverTimestamp and interval variable t in the global scope (ie, outside all functions). If the problem still persists, I would recommend adding some console.log() calls in your JavaScript to track where/how the calls are being initiated.Noma
@Aromaticity Thanks for the feedback, but what aspect did not work? The page did not load? The javascript raised an error? The Console.log did not output to console? You need to provide some further information.Noma
I meant, It was the same. I tried that too before you post the answer, I think I have to code it from scratch.Aromaticity
H
0

In your network tab if you reproduce the issue, check the request that showed the entry twice and inspect the duration, and you will see that the response took more than 1 second, which is the '1000' timeout in your javascript, therefore, I don't see any need to use that timeout.

All the requests that worked fine and showed the entry only one time, they should have gotten the server response just before '1000' (1 second) you can inspect that in your network tab by hovering the timeline attribute:

enter image description here

Or by clicking in a specific request and switching to 'Timing' tab:

enter image description here

So based on your code below is the scenario that leads to display the entry twice:

  1. ajax request
  2. server response exceeds 1 second
  3. javascript timer relaunches the same request.
  4. server responds request (1), javascript stops the timer.
  5. server responds request (2)
Hillard answered 15/10, 2014 at 16:18 Comment(5)
So should I change the timeout seconds?Aromaticity
@Aromaticity Why are you using a timeout at all is there a reason ? if you are trying to check on errors, You should check on the status of the response from the server, (you can using .error(data){//*error handling logic here */}) if there was an error (connection issue, server issue etc) then you should recall the same request again, still you need to set a limit should it stop making the same request after 5 attempts ? 10 attempts? it all depends on your logic.Hillard
@Aromaticity Any progress on this issue ? did you try what I suggested ?Hillard
Actually I coded it again from scratch and I got rid of the dirty code, It works now.Aromaticity
@Aromaticity So none of the below answers fixed your issue ? or helped you to fix it?Hillard
H
0

I can't really test this at the moment, but the biggest issues I see are

  1. the scope in which you define your interval variable t
  2. the way you pass the timestamp around
  3. the point in time where you set and clear your interval
  4. your inconsistent use of setTimeout and clearInterval

I am going to write abbreviated code, mainly to keep it conceptual. The biggest comment I can make is don't use intervals as the AJAX call might take longer than your interval. You just set a new timeout every time the ajax gets done.

// create a closure around everything
(function() {
    var timer,
        lastTimeStamp = <?php echo some_timestamp; ?>;

    function getNewPosts() {
        if(timer) clearTimeout(timer);

        $.ajax({
            data: lastTimeStamp
            // other values here
        })
        .done(function() {
            if( there_is_data ) {
                lastTimeStamp = data.timestamp;
                // render your data here
            }
        })
        .always(function() {
            timer = setTimeout(getNewPosts, 1000);
        });
    }

    $(document).ready(function() {
        // don't do your first ajax call, just let getNewPosts do it below

        // define your form submit code here

        getNewPosts();
    });
}());
Hardnett answered 16/10, 2014 at 2:29 Comment(0)
S
0

Use Java-script time-stamp rather get it back from PHP.

because you are setting timestamp when PHP is executed not when javascript setTimeOut finish.

so it could be that post time and time of fetch that post inside PHP file is same and sending back that timestamp will alow you to fetch that record again as it fits under given condition.

setting timestamp after setTimeOut will give you a fresh timestamp which could not be same as your post time.

Selby answered 16/10, 2014 at 12:2 Comment(1)
Using JavaScript timestamp? What if the user is from different time-zone? The results will be different.Aromaticity
K
0

I believe that to fix this, you will need to unbind your previous form submission. I was seeing similar issues with a script I created.

// When the form is submitted
$('#post_form').unbind('submit').on('submit', function(event) {
    $.ajax({
        url: 'ajax/post.php',
        type: 'POST',
        dataType: 'JSON',
        data: $('#post_form').serialize()
    })
    .done(function(data) {
        // Reset the form values
        $('#post_form')[0].reset();
    })
    .fail(function() {
        // When there was an error
        alert('An error occured');
    })
    // Prevent the default action
    event.preventDefault();
});
Killough answered 3/11, 2014 at 4:12 Comment(0)
I
0

It might be executing twice.

May be pasted somewhere else also May be another function is calling same.

Inflection answered 3/11, 2014 at 6:50 Comment(0)
I
0
// If there was results we will append it to the post div
    if (data.message_content ==  'results') {
        // Loop through each post and output it to the screen
        $.each(data.posts, function(index, val) {
            $("<div class='post'>" + val.post_content + "<div class='post_datePosted'>"+ val.posted_date +"</div> <br>" + "</div>").prependTo('.posts');
        });
    }

i think before prepend data to ('.post') you must clear previous data.

eg. first time : ajax result is post1

second time : ajax result is post1 + post2

--> the result of .prependTo('.posts') is post1 + post1 + post2

Ihab answered 6/11, 2014 at 2:46 Comment(0)

© 2022 - 2024 — McMap. All rights reserved.