Does readlines() return a list or an iterator in Python 3?

Question

I've read in "Dive into Python 3" that:

"The readlines() method now returns an iterator, so it is just as efficient as xreadlines() was in Python 2".

See: Appendix A: Porting Code to Python 3 with 2to3: A.26 xreadlines() I/O method.

I'm not sure that's true because they don't mention it here: http://docs.python.org/release/3.0.1/whatsnew/3.0.html . How can I check that?

Aluminize

Answer 1

Like this:

Python 3.1.2 (r312:79149, Mar 21 2010, 00:41:52) [MSC v.1500 32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> f = open('/junk/so/foo.txt')
>>> type(f.readlines())
<class 'list'>
>>> help(f.readlines)
Help on built-in function readlines:

readlines(...)
    Return a list of lines from the stream.

    hint can be specified to control the number of lines read: no more
    lines will be read if the total size (in bytes/characters) of all
    lines so far exceeds hint.

>>>

Ennis

Answer 2

The readlines method doesn't return an iterator in Python 3, it returns a list

Help on built-in function readlines:

readlines(...)
    Return a list of lines from the stream.

To check, just call it from an interactive session - it will return a list, rather than an iterator:

>>> type(f.readlines())
<class 'list'>

Dive into Python appears to be wrong in this case.

---

xreadlines has been deprecated since Python 2.3 when file objects became their own iterators. The way to get the same efficiency as xreadlines is instead of using

 for line in f.xreadlines():

you should use simply

 for line in f:

This gets you the iterator that you want, and helps to explain why readlines didn't need to change its behaviour in Python 3 - it can still return a full list, with the line in f idiom giving the iterative approach, and the long-deprecated xreadlines has been removed completely.

Eiffel

I would expect that anybody who was using for line in f.xreadlines(): will have converted it to for line in f: years ago. – Ennis

@John Machin: I'd hope so too (though perhaps not expect it!) I didn't mean to imply that xreadlines was the preferred way to do it in Python 2 - it was deprecated way back in Python 2.3 when the file-as-iterator method was introduced. I just meant to give the reason it could be removed in Python 3. – Eiffel

I only discovered files were their own iterators about a week ago :( I guess that serves me right for not looking at the tutorial/release notes since 1.5. – Alderson

This answer is better because it also provides the answer to the implicit question: how do you get an iterator on files? – Chuckchuckfull

Answer 3

Others have said as much already, but just to drive the point home, ordinary file objects are their own iterators. So having readlines() return an iterator would be silly, because it would just return the file you called it on. You can use a for loop to iterate over a file, like Scott said, and you can also pass them straight to itertools functions:

from itertools import islice
f = open('myfile.txt')
oddlines = islice(f, 0, None, 2)
firstfiveodd = islice(oddlines, 5)
for line in firstfiveodd:
  print(line)

Teplica

Not entirely silly, because the hint argument of readlines (which stops reading new lines after hint chars/bytes have been read) can be useful when working with very large files. And that is also exactly when you want an iterator rather than a list. – Ibsen