Read a .csv into pandas from F: drive on Windows 7

Question

I have a .csv file on my F: drive on Windows 7 64-bit that I'd like to read into pandas and manipulate.

None of the examples I see read from anything other than a simple file name (e.g. 'foo.csv').

When I try this I get error messages that aren't making the problem clear to me:

import pandas as pd

trainFile = "F:/Projects/Python/coursera/intro-to-data-science/kaggle/data/train.csv"
trainData = pd.read_csv(trainFile)

The error message says:

IOError: Initializing from file failed

I'm missing something simple here. Can anyone see it?

Update:

I did get more information like this:

import csv

if __name__ == '__main__':
    trainPath = 'F:/Projects/Python/coursera/intro-to-data-science/kaggle/data/train.csv'
    trainData = []
    with open(trainPath, 'r') as trainCsv:
        trainReader = csv.reader(trainCsv, delimiter=',', quotechar='"')
        for row in trainReader:
            trainData.append(row)
    print trainData

I got a permission error on read. When I checked the properties of the file, I saw that it was read-only. I was able to read 892 lines successfully after unchecking it.

Now pandas is working as well. No need to move the file or amend the path. Thanks for looking.

Bilander

honestly, your best bet is to move the file...but if you don't want to do that, try using os module to change into that directory and just call `train.csv' – Basidium

have you tried providing a buffer instead of a filepath? pd.read_csv(open(trainFile)) – Intoxication

Answer 1

I cannot promise that this will work, but it's worth a shot:

import pandas as pd
import os

trainFile = "F:/Projects/Python/coursera/intro-to-data-science/kaggle/data/train.csv"

pwd = os.getcwd()
os.chdir(os.path.dirname(trainFile))
trainData = pd.read_csv(os.path.basename(trainFile))
os.chdir(pwd)

Oriya

@RyanSaxe I'm pretty sure the thing you suggested is not the same as this and will, in fact, not work. – Oriya

how is it not? you use os.chdir to change into the directory of the file and execute it... – Basidium

Answer 2

A better solution is to use literal strings like r'pathname\filename' rather than 'pathname\filename'. See Lexical Analysis for more details.

Binocular

Answer 3

This happens to me quite often. Usually I open the csv file in Excel, and save it as an xlsx file, and it works.

so instead of

df = pd.read_csv(r"...\file.csv")

Use:

df = pd.read_excel(r"...\file.xlsx")

Habitant

Answer 4

I also got the same issue and got that resolved .

Check your path for the file correctly

I initially had the path like

dfTrain = pd.read_csv("D:\\Kaggle\\labeledTrainData.tsv",header=0,delimiter="\t",quoting=3)

This returned an error because the path was wrong .Then I have changed the path as below.This is working fine.

dfTrain = dfTrain = pd.read_csv("D:\\Kaggle\\labeledTrainData.tsv\\labeledTrainData.tsv",header=0,delimiter="\t",quoting=3)

This is because my earlier path was not correct.Hope you get it reolved

Oza

I'm getting confused by your corrected code. It seems that you typed dfTrain = and \\labeledTrainData.tsv twice, but if you deleted all these two, the initial path is the same as this one. Am I missing something here? – Reginaldreginauld

Answer 5

If you're sure the path is correct, make sure no other programs have the file open. I got that error once, and closing the Excel file made the error go away.

Lebron

Answer 6

Try this:

import os
import pandas as pd


trainFile = os.path.join('F:',os.sep,'Projects','Python','coursera','intro-to-data-science','train.csv' )
trainData = pd.read_csv(trainFile)

Oligocene