What is the difference between range and xrange functions in Python 2.X?

Question

Apparently xrange is faster but I have no idea why it's faster (and no proof besides the anecdotal so far that it is faster) or what besides that is different about

for i in range(0, 20):
for i in xrange(0, 20):

Doehne

Answer 1

In Python 2.x:

• range creates a list, so if you do range(1, 10000000) it creates a list in memory with 9999999 elements.

• xrange is a sequence object that evaluates lazily.

In Python 3:

• range does the equivalent of Python 2's xrange. To get the list, you have to explicitly use list(range(...)).
• xrange no longer exists.

Weirdo

Why did they make xrange, rather than making range lazy? – Koa

@RobertGrant If you iterate over that list 1000 times, it'll be slower to generate the values each time – Pole

@RobertGrant, they did. In Python 3. (They couldn't do that in the Python 2.x line, since all changes must be backwards compatible.) – Overstreet

Can someone explain what "evaluates lazily" means? Thanks! – Quitt

@Ratul it means that each i is evaluated on demand rather than on initialization. – Carthage

xrange is an iterator that evaluates lazily? Calling the iter() method on xrange returns a generator? – Psychomancy

@DarshanChaudhary: xrange is an iterable (with sequence-like behaviors) that evaluates lazily, not an iterator. Calling iter() on it creates an iterator (which people loosely call a generator, though in Python, "generator" is a specific type of iterator made with a generator expression or a function using the yield keyword, neither of which actually applies to xrange). – Tourmaline

To use xrange in py3 and have py2 backwards compatibility use: from past.builtins import xrange – Cherubini

Answer 2

range creates a list, so if you do range(1, 10000000) it creates a list in memory with 9999999 elements.

xrange is a generator, so it is a sequence object is a that evaluates lazily.

This is true, but in Python 3, range() will be implemented by the Python 2 xrange(). If you need to actually generate the list, you will need to do:

list(range(1,100))

Berar

I don't see that being a huge problem (regarding breaking existing applications) as range was mostly for generating indexes to be used in for loops as "for i in range(1, 10):" – Julejulee

+1 Thanks for this answer, the information about Python 3 replacing range with xrange is very useful. I actually told someone to use xrange instead or range and they said that it did not matter in python 3, so I google searched for more information and this answer came up :) – Hearsay

What is wrong with calling xrange a generator? It is a function containing yield statement, and according to glossary such functions are called generators. – Tumescent

@winterlight, think the correct term for it is iterator. Generators should be able to receive as well. – Heterogenesis

@Heterogenesis - winterlight is correct that a function that returns a generator is also referred to as a generator. A generator object is a special case of iterator. See generators vs iterators. – Goosefish

@scign, see PEP 342 for the canonical definition of the generator protocol. A nice sum up can be found in type annotation documentation (these are aliased as typing.*). – Heterogenesis

@Heterogenesis glad you agree. – Goosefish

No I don't, @scign. Have you read the linked PEP and doc? In the past the two terms might have been used interchangeably, but at the time of writing, generators must be able to receive values. In addition, Python 3 range is not an iterator either (try next(range(42))). – Heterogenesis

Answer 3

Remember, use the timeit module to test which of small snippets of code is faster!

$ python -m timeit 'for i in range(1000000):' ' pass'
10 loops, best of 3: 90.5 msec per loop
$ python -m timeit 'for i in xrange(1000000):' ' pass'
10 loops, best of 3: 51.1 msec per loop

Personally, I always use range(), unless I were dealing with really huge lists -- as you can see, time-wise, for a list of a million entries, the extra overhead is only 0.04 seconds. And as Corey points out, in Python 3.0 xrange() will go away and range() will give you nice iterator behavior anyway.

Lyns

+1 for timeit example. Note: to run in windows cmd it is needed to use double quote, i.e. ". So code will be python -m timeit "for i in xrange(1000000):" " pass" – Sasha

The main benefit of xrange is memory, not time. – Excruciate

+1 for the practical answer: use range unless huge. BTW they are conceptually identical, correct? Oddly no answer spells that out. – Ningpo

If xrange is faster and doesn't hog memory, why ever use range? – Clareta

I agree with your statement generally, but your evaluation is wrong: the extra overhead is only 0.04 seconds isnt the correct way to look at it, (90.5-51.1)/51.1 = 1.771 times slower is correct because it conveys that if this is the core loop of your program it can potentially bottleneck it. However, if this is a small part then 1.77x isnt much. – Zugzwang

This is bad advice. Really the only upside to range is if you have some aesthetic opinion on it being less ugly to read than xrange, or if you actually need a list. Additionally, running timeit on your machine is likely not going to resemble the environments in which your code is going to be running, and does nothing to tell you about the memory implications. – Aras

Answer 4

xrange only stores the range params and generates the numbers on demand. However the C implementation of Python currently restricts its args to C longs:

xrange(2**32-1, 2**32+1)  # When long is 32 bits, OverflowError: Python int too large to convert to C long
range(2**32-1, 2**32+1)   # OK --> [4294967295L, 4294967296L]

Note that in Python 3.0 there is only range and it behaves like the 2.x xrange but without the limitations on minimum and maximum end points.

Kwakiutl

Answer 5

xrange returns an iterator and only keeps one number in memory at a time. range keeps the entire list of numbers in memory.

Inessential

xrange does not return an iterator. – Salicin

and only keeps one number in memory at a time and where the rest are placed please guide me.. – Biceps

@SIslam If it knows the start, end, and current, it can compute the next, one at a time. – Aman

Answer 6

Do spend some time with the Library Reference. The more familiar you are with it, the faster you can find answers to questions like this. Especially important are the first few chapters about builtin objects and types.

The advantage of the xrange type is that an xrange object will always take the same amount of memory, no matter the size of the range it represents. There are no consistent performance advantages.

Another way to find quick information about a Python construct is the docstring and the help-function:

print xrange.__doc__ # def doc(x): print x.__doc__ is super useful
help(xrange)

Orling

The library is good but it's not always so easy to get the answer to the question you have. – Doehne

Go to the library reference, hit ctrl+f, search for range and you will get two results. It's not much effort to find the answer to this question. – Walloper

The library reference is not working. Can you please update it? – Tuneless

Answer 7

From the docs:

This function is very similar to range(), but returns an xrange object instead of a list. This is an opaque sequence type which yields the same values as the corresponding list, without actually storing them all simultaneously. The advantage of xrange() over range() is minimal (since xrange() still has to create the values when asked for them) except when a very large range is used on a memory-starved machine or when all of the range’s elements are never used (such as when the loop is usually terminated with break).

Prairial

Answer 8

You will find the advantage of xrange over range in this simple example:

import timeit

t1 = timeit.default_timer()
a = 0
for i in xrange(1, 100000000):
    pass
t2 = timeit.default_timer()

print "time taken: ", (t2-t1)  # 4.49153590202 seconds

t1 = timeit.default_timer()
a = 0
for i in range(1, 100000000):
    pass
t2 = timeit.default_timer()

print "time taken: ", (t2-t1)  # 7.04547905922 seconds

The above example doesn't reflect anything substantially better in case of xrange.

Now look at the following case where range is really really slow, compared to xrange.

import timeit

t1 = timeit.default_timer()
a = 0
for i in xrange(1, 100000000):
    if i == 10000:
        break
t2 = timeit.default_timer()

print "time taken: ", (t2-t1)  # 0.000764846801758 seconds

t1 = timeit.default_timer()
a = 0
for i in range(1, 100000000):
    if i == 10000:
        break
t2 = timeit.default_timer() 

print "time taken: ", (t2-t1)  # 2.78506207466 seconds

With range, it already creates a list from 0 to 100000000(time consuming), but xrange is a generator and it only generates numbers based on the need, that is, if the iteration continues.

In Python-3, the implementation of the range functionality is same as that of xrange in Python-2, while they have done away with xrange in Python-3

Happy Coding!!

Vanwinkle

Answer 9

range creates a list, so if you do range(1, 10000000) it creates a list in memory with 10000000 elements. xrange is a generator, so it evaluates lazily.

This brings you two advantages:

1. You can iterate longer lists without getting a MemoryError.
2. As it resolves each number lazily, if you stop iteration early, you won't waste time creating the whole list.

Arresting

Answer 10

It is for optimization reasons.

range() will create a list of values from start to end (0 .. 20 in your example). This will become an expensive operation on very large ranges.

xrange() on the other hand is much more optimised. it will only compute the next value when needed (via an xrange sequence object) and does not create a list of all values like range() does.

Josphinejoss

Answer 11

range(): range(1, 10) returns a list from 1 to 10 numbers & hold whole list in memory.

xrange(): Like range(), but instead of returning a list, returns an object that generates the numbers in the range on demand. For looping, this is lightly faster than range() and more memory efficient. xrange() object like an iterator and generates the numbers on demand.(Lazy Evaluation)

In [1]: range(1,10)

Out[1]: [1, 2, 3, 4, 5, 6, 7, 8, 9]

In [2]: xrange(10)

Out[2]: xrange(10)

In [3]: print xrange.__doc__

xrange([start,] stop[, step]) -> xrange object

Gwennie

Answer 12

range(x,y) returns a list of each number in between x and y if you use a for loop, then range is slower. In fact, range has a bigger Index range. range(x.y) will print out a list of all the numbers in between x and y

xrange(x,y) returns xrange(x,y) but if you used a for loop, then xrange is faster. xrange has a smaller Index range. xrange will not only print out xrange(x,y) but it will still keep all the numbers that are in it.

[In] range(1,10)
[Out] [1, 2, 3, 4, 5, 6, 7, 8, 9]
[In] xrange(1,10)
[Out] xrange(1,10)

If you use a for loop, then it would work

[In] for i in range(1,10):
        print i
[Out] 1
      2
      3
      4
      5
      6
      7
      8
      9
[In] for i in xrange(1,10):
         print i
[Out] 1
      2
      3
      4
      5
      6
      7
      8
      9

There isn't much difference when using loops, though there is a difference when just printing it!

Information

Answer 13

Some of the other answers mention that Python 3 eliminated 2.x's range and renamed 2.x's xrange to range. However, unless you're using 3.0 or 3.1 (which nobody should be), it's actually a somewhat different type.

As the 3.1 docs say:

Range objects have very little behavior: they only support indexing, iteration, and the len function.

However, in 3.2+, range is a full sequence—it supports extended slices, and all of the methods of collections.abc.Sequence with the same semantics as a list.^*

And, at least in CPython and PyPy (the only two 3.2+ implementations that currently exist), it also has constant-time implementations of the index and count methods and the in operator (as long as you only pass it integers). This means writing 123456 in r is reasonable in 3.2+, while in 2.7 or 3.1 it would be a horrible idea.

---

_{* The fact that issubclass(xrange, collections.Sequence) returns True in 2.6-2.7 and 3.0-3.1 is a bug that was fixed in 3.2 and not backported.}

Salicin

Answer 14

In python 2.x

range(x) returns a list, that is created in memory with x elements.

>>> a = range(5)
>>> a
[0, 1, 2, 3, 4]

xrange(x) returns an xrange object which is a generator obj which generates the numbers on demand. they are computed during for-loop(Lazy Evaluation).

For looping, this is slightly faster than range() and more memory efficient.

>>> b = xrange(5)
>>> b
xrange(5)

Migration

xrange() isn't a generator. xrange(n).__iter__()` is. – Pleader

Answer 15

When testing range against xrange in a loop (I know I should use timeit, but this was swiftly hacked up from memory using a simple list comprehension example) I found the following:

import time

for x in range(1, 10):

    t = time.time()
    [v*10 for v in range(1, 10000)]
    print "range:  %.4f" % ((time.time()-t)*100)

    t = time.time()
    [v*10 for v in xrange(1, 10000)]
    print "xrange: %.4f" % ((time.time()-t)*100)

which gives:

$python range_tests.py
range:  0.4273
xrange: 0.3733
range:  0.3881
xrange: 0.3507
range:  0.3712
xrange: 0.3565
range:  0.4031
xrange: 0.3558
range:  0.3714
xrange: 0.3520
range:  0.3834
xrange: 0.3546
range:  0.3717
xrange: 0.3511
range:  0.3745
xrange: 0.3523
range:  0.3858
xrange: 0.3997 <- garbage collection?

Or, using xrange in the for loop:

range:  0.4172
xrange: 0.3701
range:  0.3840
xrange: 0.3547
range:  0.3830
xrange: 0.3862 <- garbage collection?
range:  0.4019
xrange: 0.3532
range:  0.3738
xrange: 0.3726
range:  0.3762
xrange: 0.3533
range:  0.3710
xrange: 0.3509
range:  0.3738
xrange: 0.3512
range:  0.3703
xrange: 0.3509

Is my snippet testing properly? Any comments on the slower instance of xrange? Or a better example :-)

Homogenetic

Running a benchmark like this, one time, doesnt provide exact timing results. There is always a variance.. It could be either GC, or another process stealing the CPU... anything. That's why benchmarks are usually run 10-100-1000-... – Pleura

this is just a hasty snippet printout - I ran it a few times, but only up to around 100, and xrange seemed slightly quicker, although with Python 3 the comparison is now redundant. – Homogenetic

This is what timeit is for. It takes care of running many times, disabling GC, using the best clock instead of time, etc. – Salicin

Answer 16

xrange() and range() in python works similarly as for the user , but the difference comes when we are talking about how the memory is allocated in using both the function.

When we are using range() we allocate memory for all the variables it is generating, so it is not recommended to use with larger no. of variables to be generated.

xrange() on the other hand generate only a particular value at a time and can only be used with the for loop to print all the values required.

Gragg

Answer 17

range generates the entire list and returns it. xrange does not -- it generates the numbers in the list on demand.

Crocodile

Answer 18

xrange uses an iterator (generates values on the fly), range returns a list.

Lazulite

Answer 19

What?
range returns a static list at runtime.
xrange returns an object (which acts like a generator, although it's certainly not one) from which values are generated as and when required.

When to use which?

• Use xrange if you want to generate a list for a gigantic range, say 1 billion, especially when you have a "memory sensitive system" like a cell phone.
• Use range if you want to iterate over the list several times.

PS: Python 3.x's range function == Python 2.x's xrange function.

Cantor

xrange does not return a generator object. – Salicin

If I understand correctly, that is how it is explained here(for Python 2.x): wiki.python.org/moin/Generators – Cantor

Then the wiki is wrong. (I don't know who the "SH" is who added and signed that comment.) The official documentation is right; you can test it yourself and see whether it's a generator or a sequence. – Salicin

ok. But it's still confusing after reading this: #135541 – Cantor

If you read even just the first sentence of that comment, it says "xrange(1000) is an object that acts like a generator (although it certainly is not one)." (emphasis mine) But even if it didn't say that, are you going to trust a throwaway comment by a random SO user over the official Python documentation, or over what you can see by testing it yourself? – Salicin

At any rate, it really doesn't act like a generator at all. A sequence is a thing that can be iterated repeatedly, because its __iter__ returns a new object, and it can be indexed, because it has a __getitem__, and so on. A generator is a thing that's consumed as its iterated, because its __iter__ returns self, and it can't be indexed, and it contains a suspended stack frame, and it has send and throw methods. Which one of those does xrange sound like? – Salicin

The fun question is what to do when the interpreter disagrees with the official docs, or with a different interpreter… But fortunately, that doesn't come up too often… – Salicin

Answer 20

Everyone has explained it greatly. But I wanted it to see it for myself. I use python3. So, I opened the resource monitor (in Windows!), and first, executed the following command first:

a=0
for i in range(1,100000):
    a=a+i

and then checked the change in 'In Use' memory. It was insignificant. Then, I ran the following code:

for i in list(range(1,100000)):
    a=a+i

And it took a big chunk of the memory for use, instantly. And, I was convinced. You can try it for yourself.

If you are using Python 2X, then replace 'range()' with 'xrange()' in the first code and 'list(range())' with 'range()'.

Mooneyham

Answer 21

From the help docs.

Python 2.7.12

>>> print range.__doc__
range(stop) -> list of integers
range(start, stop[, step]) -> list of integers

Return a list containing an arithmetic progression of integers.
range(i, j) returns [i, i+1, i+2, ..., j-1]; start (!) defaults to 0.
When step is given, it specifies the increment (or decrement).
For example, range(4) returns [0, 1, 2, 3].  The end point is omitted!
These are exactly the valid indices for a list of 4 elements.

>>> print xrange.__doc__
xrange(stop) -> xrange object
xrange(start, stop[, step]) -> xrange object

Like range(), but instead of returning a list, returns an object that
generates the numbers in the range on demand.  For looping, this is 
slightly faster than range() and more memory efficient.

Python 3.5.2

>>> print(range.__doc__)
range(stop) -> range object
range(start, stop[, step]) -> range object

Return an object that produces a sequence of integers from start (inclusive)
to stop (exclusive) by step.  range(i, j) produces i, i+1, i+2, ..., j-1.
start defaults to 0, and stop is omitted!  range(4) produces 0, 1, 2, 3.
These are exactly the valid indices for a list of 4 elements.
When step is given, it specifies the increment (or decrement).

>>> print(xrange.__doc__)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
NameError: name 'xrange' is not defined

Difference is apparent. In Python 2.x, range returns a list, xrange returns an xrange object which is iterable.

In Python 3.x, range becomes xrange of Python 2.x, and xrange is removed.

Wantage

Answer 22

range() in Python 2.x

This function is essentially the old range() function that was available in Python 2.x and returns an instance of a list object that contains the elements in the specified range.

However, this implementation is too inefficient when it comes to initialise a list with a range of numbers. For example, for i in range(1000000) would be a very expensive command to execute, both in terms of memory and time usage as it requires the storage of this list into the memory.

---

range() in Python 3.x and xrange() in Python 2.x

Python 3.x introduced a newer implementation of range() (while the newer implementation was already available in Python 2.x through the xrange() function).

The range() exploits a strategy known as lazy evaluation. Instead of creating a huge list of elements in range, the newer implementation introduces the class range, a lightweight object that represents the required elements in the given range, without storing them explicitly in memory (this might sound like generators but the concept of lazy evaluation is different).

---

As an example, consider the following:

# Python 2.x
>>> a = range(10)
>>> type(a)
<type 'list'>
>>> b = xrange(10)
>>> type(b)
<type 'xrange'>

and

# Python 3.x
>>> a = range(10)
>>> type(a)
<class 'range'>

Collateral

Answer 23

On a requirement for scanning/printing of 0-N items , range and xrange works as follows.

range() - creates a new list in the memory and takes the whole 0 to N items(totally N+1) and prints them. xrange() - creates a iterator instance that scans through the items and keeps only the current encountered item into the memory , hence utilising same amount of memory all the time.

In case the required element is somewhat at the beginning of the list only then it saves a good amount of time and memory.

Oran

xrange does not create an iterator instance. It creates an xrange object, which is iterable, but not an iterator—almost (but not quite) a sequence, like a list. – Salicin

Answer 24

Range returns a list while xrange returns an xrange object which takes the same memory irrespective of the range size,as in this case,only one element is generated and available per iteration whereas in case of using range, all the elements are generated at once and are available in the memory.

Dufresne

Answer 25

The difference decreases for smaller arguments to range(..) / xrange(..):

$ python -m timeit "for i in xrange(10111):" " for k in range(100):" "  pass"
10 loops, best of 3: 59.4 msec per loop

$ python -m timeit "for i in xrange(10111):" " for k in xrange(100):" "  pass"
10 loops, best of 3: 46.9 msec per loop

In this case xrange(100) is only about 20% more efficient.

Pocket

Answer 26

range :-range will populate everything at once.which means every number of the range will occupy the memory.

xrange :-xrange is something like generator ,it will comes into picture when you want the range of numbers but you dont want them to be stored,like when you want to use in for loop.so memory efficient.

Liquidator

Answer 27

Additionally, if do list(xrange(...)) will be equivalent to range(...).

So list is slow.

Also xrange really doesn't fully finish the sequence

So that's why its not a list, it's a xrange object

Viridi

Answer 28

See this post to find difference between range and xrange:

To quote:

range returns exactly what you think: a list of consecutive integers, of a defined length beginning with 0. xrange, however, returns an "xrange object", which acts a great deal like an iterator

Serpentine

I realize this is 5 years old, but that post is wrong about nearly everything. xrange is not an iterator. The list returned by range does support iteration (a list is pretty much the prototypical example of an iterable). The overall benefit of xrange is not "minimal". And so on. – Salicin