How to stop papaparse streaming after some results

Question

I am using PapaPase to parse big CSV file, using chunk mode.

I am validating csv data, and I want to stop streaming when validation fails.

But I am unable to stop streaming, after some parsing.

I tried to stop using return false from chunk callback, but it's not working.

Below is the code.

$("#fileselect").on("change", function(e){
    if (this.files.length) {
        var file = this.files[0]
        count = 0;
        Papa.parse(file, {
            worker: true,
            delimiter: "~~",
            skipEmptyLines:true,
            chunk: function (result) {
                count += result.data.length;
                console.clear();
                console.log(count);
                if (count>60000) {
                    return false;
                }
            },
            complete: function (result, file) {
                console.log(result)
            }
        });
    }
});

Clawson

any luck with this? I'm trying the same... – Madness

Answer 1

Chunk, and Step, both have access to parser too, you can use that to pause, resume, or (as you might want) abort.

step: function(results, parser) {
console.log("Row data:", results.data);
console.log("Row errors:", results.errors);
}’

So in your instance, you would need to do this (untested):

$("#fileselect").on("change", function(e){
    if (this.files.length) {
        var file = this.files[0]
        count = 0;
        Papa.parse(file, {
            worker: true,
            delimiter: "~~",
            skipEmptyLines:true,
            chunk: function (result, parser) {
                count += result.data.length;
                console.clear();
                console.log(count);
                if (count>60000) {
                    //return false;
                    parser.abort(); // <-- stop streaming
                }
            },
            complete: function (result, file) {
                console.log(result)
            }
        });
    }
});

Take a look at the documentation for step and chunk.

https://www.papaparse.com/docs

Hope this helped!

Edile

Answer 2

In my case, I just needed the first 10 rows of data from a file. If anyone needs a solution for this, here's an example of how I got it to work:

In order to stop streaming after a number of rows, simply pass in the 'preview' option in the configs.

let fileInput = document.getElementById('myFile');
let file = fileInput.files[0];
let parsedData; //variable to store the chunked results
Papa.parse(file, {
    worker: true,
    preview: 10, //this is what you need to do the trick,
    chunk: function(results){
       parsedData = results; //set results to the parsedData variable.
       //I'm doing this because "When streaming, parse results are not available in 
       the 'complete' callback."
    },
    complete: function(){
       console.log(parsedData); //log the results once parsing is completed
        /**
         Do whatever else you want with parsedData here.
         In my case, I just created an html table to show a preview of the data.
        */

    }
});

with this, you should be able to parse very large files without crashing the browser. I tested with a .csv file that has over 1 million rows and I got no issues.

see the docs: https://www.papaparse.com/docs#config-details

Inclining