Skip first rows when writing csv (pandas.DataFrame.to_csv)

Question

In my python script I am reading a csv file via

df = pd.read_csv('input.csv', sep=';', encoding = "ISO-8859-1", skiprows=2, skipfooter=1, engine='python')

I am the skipping the first two rows in the csv file because they are just discriptions I don't need.

After importing, I am filtering and separating the data. I want to write the data back to csv files while having the same format as before (first two rows either empty or the description as before the import). How can I do that?

Currently I am using

df.to_csv('output.csv'), sep=';', encoding = "ISO-8859-1")

Is there something like a parameter "skiprows" for exporting? I can't find one in the api documentation for .to_csv.

Remand

Could you not write your description first and then append to the file? Or just write 2 new lines and then append? There isn't a built in api in pandas for this – Proctor

Maybe you are right. I see you can use mode='a' for appending when writing csv. So I just need to create a csv file with two empty rows before exporting. – Remand

Appending to a pre-existing file is trivial, there isn't a method built in pandas to do this using to_csv – Proctor

Answer 1

One possible solution is write DataFrame with NaNs first and then append original DataFrame:

df1 = pd.DataFrame({'a':[np.nan] * 2})
df1.to_csv('output.csv', index=False, header=None)
df.to_csv('output.csv', sep=';', encoding = "ISO-8859-1", mode='a')

Or same original header to df1 and this write first, only necessary no value | in header data:

df1 = pd.read_csv('input.csv', sep='|', encoding = "ISO-8859-1", nrows=2, names=['tmp'])

df1.to_csv('output.csv', index=False, header=None)
df.to_csv('output.csv', sep=';', encoding = "ISO-8859-1", mode='a')

Biegel

Perfect, first solution works for me. I will switch to the second one if i really need the description later. Thanks! – Remand

Thanks for this answer @jezrael, just wanted to know if we have a way to skip first row alone in .to_csv() ? – Icecap

Answer 2

You want to copy the header from your original file and write to your new file. Then you append your dataframe by setting mode to 'a'

with open("my_csv.csv") as f:
    header =''
    for line in range(0,header_length_in_lines):
        header +=f.readline()

with open('my_new_csv.csv','w+') as f:
    f.write(header)
df.to_csv('my_new_csv.csv', mode='a', index=False) 

Kathlyn

I want to export a dataframe (500 rows,2 coulmns) from python to a csv file. However, I need to ensure that 1st 20 rows has some text/strings written and then the dataframe should should start from 21st row onwards. Can somebody please let me know how do we do this? – Chuchuah