Using readlines in python? First time

Question

I have a text file with columns of data and I need to turn these columns into individual lists or arrays. This is what I have so far

f = open('data.txt', 'r')
temp = []
for row in f.readlines():
    Data = row.split()
    temp.append(float(Data[0]))

When I run this I get IndexError: list index out of range.

Snippet of the data below:

16  0.2000  
17  0.3000  
18  0.4000  
20  0.5000  
21  0.6000  
22  0.7000
24  0.8000  
25  0.9000
26  1.000   

I need the first column, if possible to look like this: Data = [16, 17, 18, 20, 21, 22, 24, 25, 26]

Glottology

csv is your friend. – Bumf

...shouldn't this be for row in f.readlines(): or more simply for row in f? – Interlay

Answer 1

You are getting an empty list Data=[] if you read an empty row. You try to get the first element from the list using Data[0],but because it's an empty list it doesn't have an element at position 0, so you get an IndexError.

Data=''.split()

Data[0]
---------------------------------------------------------------------------
IndexError                                Traceback (most recent call last)
<ipython-input-686-0792b03cbbdf> in <module>()
----> 1 Data[0]

IndexError: list index out of range

This will print out the Data if IndexError occours - you can see yourself that it prints an empty list:

f=open('file','r')
temp = []
for row in f.readlines():
    Data = row.split()
    try:
        temp.append(float(Data[0]))
    except IndexError:
        print Data

You can use the with statement to open the file, that automatically closes the file after being processed. Also you can loop over the file itself, without using readlines().

with open(file,'r') as f:        
     for row in f:
         Data = row.split()
         try:
            print Data[0]
         except IndexError:
            print 'You have an empty row'

EDIT: You are better of using the csv module:

import csv
with open('file.csv', 'rb') as f:
    reader = csv.reader(f, delimiter=' ')
    print [row[0] for row in reader if len(row)]
>>> 
['16', '17', '18', '20', '21', '22', '24', '25', '26']

Octofoil

Thanks very much for your feedback and suggestions. Is there a way that I can then get the Data into a list form like this: Data = [0, 1, etc]? – Glottology

I'm just trying to get the data into a list because I need to then use it to perform a linear interpolation. If I use the last method you suggested, (using 'with') will that give me a list that I can then use later in my interpolation? The 0,1, etc were just examples of the data values being imported – Glottology

if you want to use interpolation take a look at the pandas library (it can import the data dirctly from csv), here's an interpolation example using pandas as an answer to a previous question #12983292 – Octofoil

Ok just one more question, will this work if I want to do the same for other columns from the data? – Glottology

yes it will. just change row[0] to row[1], if you have rows that have the first value and not the second you may need to add if len(row)>1 to avoid index error. – Octofoil

I tried the code and rather than giving me the values just from the first column its giving me the values from all columns? – Glottology

is the data still separated by a space? It looks like a delimeter issue. – Octofoil

Do you mean after I've run it? If so, then no its not – Glottology

No in the file. delimiter='' breaks the rows by whitspace, if it is something else that separates the values, it wont work. – Octofoil

Oh ok, yeah I just checked and there is no spaces. I will put spaces and see if it works – Glottology

Answer 2

use with for filehandlers.

with open('path/to/file', 'r') as f:
    for line in f:
        # code.

you index error comes from trying to access Data at the [0] location. which simply means your line was empty.

you should run a quick check before parsing the line...

if len(Data):
    #code
else:
    #empty line?

Arterial

In Python, it is better to write if len(Data): and not if len(Data) > 0:. – Bumf

@ Burhan Khalid -- cant you then just do if Data:? – Octofoil

@Octofoil no, because an empty list still exists. >>> a=[] >>> print a [] – Arterial

@ Inbar Rose -- >>> if a: ... print 'foo' ... else: print 'bar' ... bar – Octofoil

Answer 3

f = open('data.txt', 'r')
temp = []
for row in f.readlines():
    items = row.split(',')
    temp.append(unicode(items[0]))

I hope that it solves your problem.

Lingam

Answer 4

def cal(num_list):
    x = 1;
    z = 0;
    while True:
        list1 = list(str(x))
        list2 = [int(a) for a in list1]

        for i in range(len(list2)):
            for j in range(10):
                if(list2.count(j) > num_list[list2[i]]):
                    z = 1;
                    break;
        if(z == 1):   
            save(x);
            break;      
        x = x + 1;

Salvador

Answer 5

#matrix A to B
def show():
    global string_final,list_2,tot
    index=matrix_1.index(num)  # finding index of num

    length=n
    j=(index)%length # the column positon
    i=index/length # the row position

    list_1=[]
    for a in range(length):
        lis=[]
        for b in range(length):

            lis.append(matrix_1.pop(0)) #pop the first element and append the list

        list_1.append(lis)

    tot=0
    list_2=[]

    for a in range(i+1):
        for b in range(j+1):
            tot=tot+list_1[a][b] # add the numbers
            list_2.append(list_1[a][b]) #append to list
            if(b!=length):
                list_2.append(" ")  #append space
        list_2.append("\n")
    string_final="".join([str(a) for a in list_2])  #list to string


    print "matrix B\n",string_final,"\nx: ",str(num),"\nn: ",str(n)+"\ntotal: "+str(tot) # print the result

Dilisio

#mat A to b def saveFile(): file=open("matrix2.txt","w") file.write("matrixB\n"+string_final+"\nx: "+str(num)+"\nn: "+str(n)+"\ntotal: "+str(tot)) file.close(); x=1 while x==1: getInput() if(n<=0 or n>=20): #check for the n print "enter correct n" break if(len(matrix_1)!=n*n or len(set(matrix_1))!=len(matrix_1)): # check the input is ok or not print "wrong input" break if(set([99>a>9 for a in matrix_1])!={True}):# check the value of n print "enter suitable number" break show() saveFile() x=x+1; – Dilisio

Comments destroy code formatting, especially in whitespace-significant languages. Please edit your answer to include the additional code. – Gurule

Answer 6

#matrix add zero
def get():
    global b_no
    num_list=[];
    file=open("matrix.txt","r");
    for i in file:
        b_no=0
        if(len(i.split())==1): #check length is 1 or not
            n=int(i)#string to int
        else:
            temp=i.split();
            if(len(temp)!=n):
                b_no+=1
                break
            temp=[0]+[int(a) for a in temp]+[0] #add zero at first and last
            num_list.append(temp);

    zero_list=[]

    for i in range(n+2):
        zero_list.append(0)         #append zero
    num_list.insert(0,zero_list)
    num_list.insert(n+1,zero_list)
    return num_list,n; 

Dilisio

#mat Add zero# def cal(): list1=[]; for row in range(1,n+1): for col in range(1,n+1): tot=0; count=0; for m in range(-1,2): for p in range(-1,2): if(num_list[row+m][col+p]==0): count=count+1; tot=tot+num_list[row+m][col+p]; if(count!=0): tot=tot/(9-count); else: tot=tot/9 list1.append(str(tot)+" "); list1.append("\n"); print "".join([str(a) for a in list1]) – Dilisio

x=1; while x==1: num_list,n=get(); if(n>20 or n<3): print "n should between 3 and 20" break if(b_no!=0): print "enter correctly" break cal() x=x+1; – Dilisio

Answer 7

I would avoid using Data[0] or row[0], instead, I would use ''.join(Data) or ''.join(row) to avoid empty list (list index out of range error).

Mycetozoan