How can I read a local file with Papa Parse?

Question

How can I read a local file with Papa Parse? I have a file locally called challanges.csv, but after many tried I can't parse it with Papa Parse.

var data;

Papa.parse('challanges.csv', {
  header: true,
  dynamicTyping: true,
  complete: function(results) {
    console.log(results);
    data = results.data;
  }
});

As far as I know, I'm having problems with opening the csv file as File. How can I do it with javascript?

Karyokinesis

replace 'challanges.csv' to a file that you get from a file input or some drag and drop – Asdic

This article may help – Pneumograph

thank you both, it works indeed, but now every time I refresh my browser, I have to upload the file. Isn't there an easier way? @Asdic – Karyokinesis

if you want to keep the file after refresh then you need to store the file in some storage layer (preferably the indexedDB storage) for this i can recommend: localForage – Asdic

Answer 1

The File API suggested by papaparse's docs is meant for browser used. Assuming that you are running this on node at server side, what works for me is leveraging the readable stream:

const fs = require('fs');
const papa = require('papaparse');
const file = fs.createReadStream('challenge.csv');
var count = 0; // cache the running count
papa.parse(file, {
    worker: true, // Don't bog down the main thread if its a big file
    step: function(result) {
        // do stuff with result
    },
    complete: function(results, file) {
        console.log('parsing complete read', count, 'records.'); 
    }
});

There may be an easier interface, but so far this works quite well and offer the option of streaming for processing large files.

Jacinto

This is a beautiful package! It worked well for me just loading csv in the frontend with React. – Fluctuation

@Fluctuation Did you use the code in Philip's answer to do that or did you roll your own? – Mezereon

@Casimir I used pretty much the same code as above. Why? Are you having issues? – Fluctuation

I haven't tried the above code yet. It's just that fs doesn't work client-side so I was curious how you ported it to frontend React. – Mezereon

I just passed the file array object that you usually get in React. – Fluctuation

As stated in PapaParse README, the worker config option is unavailable on Node – Rectilinear

Answer 2

None of these worked for me, I specifically wanted to read in a csv server side, and not client side (in the borwser). This worked for me.

async / await

const fs = require('fs');
const Papa = require('papaparse');

const csvFilePath = 'data/test.csv'

// Function to read csv which returns a promise so you can do async / await.

const readCSV = async (filePath) => {
  const csvFile = fs.readFileSync(filePath)
  const csvData = csvFile.toString()  
  return new Promise(resolve => {
    Papa.parse(csvData, {
      header: true,
      complete: results => {
        console.log('Complete', results.data.length, 'records.'); 
        resolve(results.data);
      }
    });
  });
};

const test = async () => {
  let parsedData = await readCSV(csvFilePath); 
}

test()

If you don't want promise / async, await then you can use this.

callback

const fs = require('fs');
const Papa = require('papaparse');

const csvFilePath = 'data/test.csv'

const file = fs.createReadStream(csvFilePath);

var csvData=[];
Papa.parse(file, {
  header: true,
  step: function(result) {
    csvData.push(result.data)
  },
  complete: function(results, file) {
    console.log('Complete', csvData.length, 'records.'); 
  }
});

---

Note header: true is an option on the config, see docs for other options

Motherhood

This is absolutely awesome. Async/Await is way to go. – Gaudet

Calling readFileSync(filePath) is a performance killer. Will block the entire application while it is reading. Worse yet, it's already in an async funciton... – Cheyney

Answer 3

You need to add one more line in your config: download: true,.

var data;

Papa.parse('../challanges.csv', {
  header: true,
  download: true,
  dynamicTyping: true,
  complete: function(results) {
    console.log(results);
    data = results.data;
  }
});

Update: with this answer you dont need a FILE OBject. You can pass the filename and papa parse will "download" it.

Supposed

I tried your suggestion, but got ...\node_modules\papaparse\papaparse.js:591 ReferenceError: XMLHttpRequest is not defined error. I'm using Papa Parse v.4.5. – Mixie

@Mixie look at here: #38644580 – Supposed

Answer 4

Here is my solution: 1: FE: Angular 10, NPM install ngx-papaparse Then import in the component

import { Papa } from "ngx-papaparse";

2: one input with type file only accept csv file on FE

 <input type="file" id="csv-file" accept=".csv" (change)="onFileSelected($event)">

3: one button to read CSV file

4: we get the file data from the onFileSelected function

    const files = event.target.files;
    if (files.length === 0)
      return;
    this.selectedFile = files[0] as File;

5: call read CSV file function

  this.papa.parse(this.selectedFile, {
        delimiter: ",",
        newline: "",
        header: true,
        dynamicTyping: true,
        complete: (result) => {
          this.fileData= result.data;
        }
      });

6: you should get the object fileData Tips: check the papaparse documentation to see what configuration you need. https://www.papaparse.com/ eg: header: true --> means you will get the headers in the result object

Balladry

Answer 5

This is to reiterate that the best answer is Murat Seker's.

All that is necessary is to add the to the config download: true and the local path will be downloaded by Papa Parse. The streaming answer by Philip M. is not the best answer.

var data;

Papa.parse('challanges.csv', {
  header: true,
  download: true,
  dynamicTyping: true,
  complete: function(results) {
    console.log(results);
    data = results.data;
  }
});

P.S. I do not have enough reputation to comment on Murat Seker's answer. So, I reposted an answer. Any love towards reputation will be appreciated. :-)

Laodicea

The parameter download: true makes accessing to the file as an URL, so it's not reading it locally, instead is trying to download it from a web page. The OP asks specifically about reading a local file. – Heti

This works for local files or web files. This is the code I use to read local files. – Laodicea

As you can read in the docs (papaparse.com/docs), if you set download: true, will use the path as an URL, and so, will try to download it (even if it's from localhost, that's not accessing a local file, it's downloading it from a local web server) – Heti

With all due respect, Alex, regardless of what the documents say, the solution as I provided works. It is exactly what is running in production in our system. – Laodicea

The solution as provided only works if a web server is running on the local machine. In your answer you do not specify it, and just executing this code without a web server will not work. Of course, also, you will be limited to read documents from the web server directory and will not be accessing the local file system. And the answer is specifically saying that wants to read a local file. – Heti

Where would a string representing a local file read from if not the local file system? You are technically correct that if the code is running client side the configuration I provide will attempt to read a file from the local file system. That is what the original question was asking. – Laodicea

From a local or remote web server, which is not the same as the local file system. If you provide the parameter download: true it won't even try to read the file system, it will make an HTTP request. – Heti

The OP asks for how to access the local file system, which is implied to be the local file system of the service or the client. The solution I present provides that solution. – Laodicea