Should you always favor xrange() over range()?

Question

Why or why not?

Ingalls

Can someone briefly describe the difference between the 2 for us non-python guys? Maybe something like "xrange() does everyhing range() does, but also supports X, Y, and Z" – Shadwell

range(n) creates a list containing all the integers 0..n-1. This is a problem if you do range(1000000), because you'll end up with a >4Mb list. xrange deals with this by returning an object that pretends to be a list, but just works out the number needed from the index asked for, and returns that. – Kappenne

See stackoverflow.com/questions/94935 – Impermissible

Basically, whereas range(1000) is a list, xrange(1000) is an object that acts like a generator (although it certainly is not one). Also, xrange is faster. You can import timeit from timeit and then make a method that just has for i in xrange: pass and another for range, then do timeit(method1) and timeit(method2) and, lo and behold, xrange is almost twice as fast sometimes (that is when you don't need a list). (For me, for i in xrange(1000):pass vs for i in range(1000):pass took 13.316725969314575 vs 21.190124988555908 seconds respectively - that's a lot.) – Steed

Another performance test gives xrange(100) as 20% faster than range(100). – Thermistor

Answer 1

For performance, especially when you're iterating over a large range, xrange() is usually better. However, there are still a few cases why you might prefer range():

• In python 3, range() does what xrange() used to do and xrange() does not exist. If you want to write code that will run on both Python 2 and Python 3, you can't use xrange().

• range() can actually be faster in some cases - eg. if iterating over the same sequence multiple times. xrange() has to reconstruct the integer object every time, but range() will have real integer objects. (It will always perform worse in terms of memory however)

• xrange() isn't usable in all cases where a real list is needed. For instance, it doesn't support slices, or any list methods.

[Edit] There are a couple of posts mentioning how range() will be upgraded by the 2to3 tool. For the record, here's the output of running the tool on some sample usages of range() and xrange()

RefactoringTool: Skipping implicit fixer: buffer
RefactoringTool: Skipping implicit fixer: idioms
RefactoringTool: Skipping implicit fixer: ws_comma
--- range_test.py (original)
+++ range_test.py (refactored)
@@ -1,7 +1,7 @@

 for x in range(20):
-    a=range(20)
+    a=list(range(20))
     b=list(range(20))
     c=[x for x in range(20)]
     d=(x for x in range(20))
-    e=xrange(20)
+    e=range(20)

As you can see, when used in a for loop or comprehension, or where already wrapped with list(), range is left unchanged.

Kappenne

What do you mean by "range will become an iterator"? Should this not be "generator"? – Duong

No. Generator refers to a specific type of iterator, and the new range isn't an iterator anyway. – Sclera

Your second bullet doesn't really make any sense. You are saying that you can't use an object multiple times; sure you can! try xr = xrange(1,11) then on the next line for i in xr: print " ".join(format(i*j,"3d") for j in xr) and voila! You have your times-tables up to ten. It works just the same as r = range(1,11) and for i in r: print " ".join(format(i*j,"3d") for j in r)... everything is an object in Python2. I think what you meant to say is that you can do index-based comprehension (if that makes sense) better with range as opposed to xrange. Range is handy very seldom, I think – Steed

(Not enough room) Although, I do think that range can be handy if you want to use a list in a loop and then change certain indices based on certain conditions or append things to that list, then range is definitely better. However, xrange is simply faster and uses less memory, so for the majority of for loop applications, it appears to be the best. There are instances - going back to the questioner's question - seldom but existent, where range would be better. Perhaps not as seldom as I am thinking, but I certainly use xrange 95% of the time. – Steed

Answer 2

No, they both have their uses:

Use xrange() when iterating, as it saves memory. Say:

for x in xrange(1, one_zillion):

rather than:

for x in range(1, one_zillion):

On the other hand, use range() if you actually want a list of numbers.

multiples_of_seven = range(7,100,7)
print "Multiples of seven < 100: ", multiples_of_seven

Interstellar

Answer 3

You should favour range() over xrange() only when you need an actual list. For instance, when you want to modify the list returned by range(), or when you wish to slice it. For iteration or even just normal indexing, xrange() will work fine (and usually much more efficiently). There is a point where range() is a bit faster than xrange() for very small lists, but depending on your hardware and various other details, the break-even can be at a result of length 1 or 2; not something to worry about. Prefer xrange().

Flitch

Answer 4

One other difference is that Python 2 implementation of xrange() can't support numbers bigger than C ints, so if you want to have a range using Python's built in large number support, you have to use range().

Python 2.7.3 (default, Jul 13 2012, 22:29:01) 
[GCC 4.7.1] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> range(123456787676676767676676,123456787676676767676679)
[123456787676676767676676L, 123456787676676767676677L, 123456787676676767676678L]
>>> xrange(123456787676676767676676,123456787676676767676679)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
OverflowError: Python int too large to convert to C long

Python 3 does not have this problem:

Python 3.2.3 (default, Jul 14 2012, 01:01:48) 
[GCC 4.7.1] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> range(123456787676676767676676,123456787676676767676679)
range(123456787676676767676676, 123456787676676767676679)

Detrain

Answer 5

xrange() is more efficient because instead of generating a list of objects, it just generates one object at a time. Instead of 100 integers, and all of their overhead, and the list to put them in, you just have one integer at a time. Faster generation, better memory use, more efficient code.

Unless I specifically need a list for something, I always favor xrange()

Vendor

Answer 6

range() returns a list, xrange() returns an xrange object.

xrange() is a bit faster, and a bit more memory efficient. But the gain is not very large.

The extra memory used by a list is of course not just wasted, lists have more functionality (slice, repeat, insert, ...). Exact differences can be found in the documentation. There is no bonehard rule, use what is needed.

Python 3.0 is still in development, but IIRC range() will very similar to xrange() of 2.X and list(range()) can be used to generate lists.

Wyman

Answer 7

I would just like to say that it REALLY isn't that difficult to get an xrange object with slice and indexing functionality. I have written some code that works pretty dang well and is just as fast as xrange for when it counts (iterations).

from __future__ import division

def read_xrange(xrange_object):
    # returns the xrange object's start, stop, and step
    start = xrange_object[0]
    if len(xrange_object) > 1:
       step = xrange_object[1] - xrange_object[0]
    else:
        step = 1
    stop = xrange_object[-1] + step
    return start, stop, step

class Xrange(object):
    ''' creates an xrange-like object that supports slicing and indexing.
    ex: a = Xrange(20)
    a.index(10)
    will work

    Also a[:5]
    will return another Xrange object with the specified attributes

    Also allows for the conversion from an existing xrange object
    '''
    def __init__(self, *inputs):
        # allow inputs of xrange objects
        if len(inputs) == 1:
            test, = inputs
            if type(test) == xrange:
                self.xrange = test
                self.start, self.stop, self.step = read_xrange(test)
                return

        # or create one from start, stop, step
        self.start, self.step = 0, None
        if len(inputs) == 1:
            self.stop, = inputs
        elif len(inputs) == 2:
            self.start, self.stop = inputs
        elif len(inputs) == 3:
            self.start, self.stop, self.step = inputs
        else:
            raise ValueError(inputs)

        self.xrange = xrange(self.start, self.stop, self.step)

    def __iter__(self):
        return iter(self.xrange)

    def __getitem__(self, item):
        if type(item) is int:
            if item < 0:
                item += len(self)

            return self.xrange[item]

        if type(item) is slice:
            # get the indexes, and then convert to the number
            start, stop, step = item.start, item.stop, item.step
            start = start if start != None else 0 # convert start = None to start = 0
            if start < 0:
                start += start
            start = self[start]
            if start < 0: raise IndexError(item)
            step = (self.step if self.step != None else 1) * (step if step != None else 1)
            stop = stop if stop is not None else self.xrange[-1]
            if stop < 0:
                stop += stop

            stop = self[stop]
            stop = stop

            if stop > self.stop:
                raise IndexError
            if start < self.start:
                raise IndexError
            return Xrange(start, stop, step)

    def index(self, value):
        error = ValueError('object.index({0}): {0} not in object'.format(value))
        index = (value - self.start)/self.step
        if index % 1 != 0:
            raise error
        index = int(index)


        try:
            self.xrange[index]
        except (IndexError, TypeError):
            raise error
        return index

    def __len__(self):
        return len(self.xrange)

Honestly, I think the whole issue is kind of silly and xrange should do all of this anyway...

Dispute

Yeah agreed; from a totally different technology, check the work on lodash to make it lazy: github.com/lodash/lodash/issues/274. Slicing etc should still be as lazy as possible and where not, only then reify. – Periostitis

Answer 8

Go with range for these reasons:

1) xrange will be going away in newer Python versions. This gives you easy future compatibility.

2) range will take on the efficiencies associated with xrange.

Ichthyo

Don't do this. xrange() will go away, but so will a lot of other things. The tool you will be using to translate your Python 2.x code into Python 3.x code will automatically translate xrange() to range(), but range() will be translated to the less efficient list(range()). – Flitch

Thomas: Its actually a bit smarter than that. It'll translate range() in situations where it clearly doesn't need a real list (eg in a for loop, or comprehension) to just plain range(). Only cases where it gets assigned to a variable, or used directly must be wrapped with list() – Kappenne

Answer 9

A good example given in book: Practical Python By Magnus Lie Hetland

>>> zip(range(5), xrange(100000000))
[(0, 0), (1, 1), (2, 2), (3, 3), (4, 4)]

I wouldn’t recommend using range instead of xrange in the preceding example—although only the first five numbers are needed, range calculates all the numbers, and that may take a lot of time. With xrange, this isn’t a problem because it calculates only those numbers needed.

Yes I read @Brian's answer: In python 3, range() is a generator anyway and xrange() does not exist.

Carruth

Answer 10

While xrange is faster than range in most circumstances, the difference in performance is pretty minimal. The little program below compares iterating over a range and an xrange:

import timeit
# Try various list sizes.
for list_len in [1, 10, 100, 1000, 10000, 100000, 1000000]:
  # Time doing a range and an xrange.
  rtime = timeit.timeit('a=0;\nfor n in range(%d): a += n'%list_len, number=1000)
  xrtime = timeit.timeit('a=0;\nfor n in xrange(%d): a += n'%list_len, number=1000)
  # Print the result
  print "Loop list of len %d: range=%.4f, xrange=%.4f"%(list_len, rtime, xrtime)

The results below shows that xrange is indeed faster, but not enough to sweat over.

Loop list of len 1: range=0.0003, xrange=0.0003
Loop list of len 10: range=0.0013, xrange=0.0011
Loop list of len 100: range=0.0068, xrange=0.0034
Loop list of len 1000: range=0.0609, xrange=0.0438
Loop list of len 10000: range=0.5527, xrange=0.5266
Loop list of len 100000: range=10.1666, xrange=7.8481
Loop list of len 1000000: range=168.3425, xrange=155.8719

So by all means use xrange, but unless you're on a constrained hardware, don't worry too much about it.

Afterheat

Your list_len is not being used and so you're only running this code for lists of length 100. – Arst

I'd recommend actually modifying the list length: rtime = timeit.timeit('a=0;\nfor n in range(%d): a += n' % list_len, number=1000) – Arst

Wow, this is shaping up to be a good week, thanks, fixed. Still, not a major difference. – Afterheat

Answer 11

Okay, everyone here as a different opinion as to the tradeoffs and advantages of xrange versus range. They're mostly correct, xrange is an iterator, and range fleshes out and creates an actual list. For the majority of cases, you won't really notice a difference between the two. (You can use map with range but not with xrange, but it uses up more memory.)

What I think you rally want to hear, however, is that the preferred choice is xrange. Since range in Python 3 is an iterator, the code conversion tool 2to3 will correctly convert all uses of xrange to range, and will throw out an error or warning for uses of range. If you want to be sure to easily convert your code in the future, you'll use xrange only, and list(xrange) when you're sure that you want a list. I learned this during the CPython sprint at PyCon this year (2008) in Chicago.

Nock

Thats not true. Code like "for x in range(20)" will be left as range, and code like "x=range(20)" will be converted to "x=list(range(20))" - no errors. Further, if you want to write code that will run under both 2.6 and 3.0, range() is your only option without adding compatability functions. – Kappenne

Answer 12

• range(): range(1, 10) returns a list from 1 to 10 numbers & hold whole list in memory.
• xrange(): Like range(), but instead of returning a list, returns an object that generates the numbers in the range on demand. For looping, this is lightly faster than range() and more memory efficient. xrange() object like an iterator and generates the numbers on demand (Lazy Evaluation).

In [1]: range(1,10)
Out[1]: [1, 2, 3, 4, 5, 6, 7, 8, 9]

In [2]: xrange(10)
Out[2]: xrange(10)

In [3]: print xrange.__doc__
Out[3]: xrange([start,] stop[, step]) -> xrange object

range() does the same thing as xrange() used to do in Python 3 and there is not term xrange() exist in Python 3. range() can actually be faster in some scenario if you iterating over the same sequence multiple times. xrange() has to reconstruct the integer object every time, but range() will have real integer objects.

Haftarah