What's the difference between heapq and PriorityQueue in python?

Question

In python there's a built-in heapq algorithm that gives you push, pop, nlargest, nsmallest... etc that you can apply to lists. However, there's also the queue.PriorityQueue class that seems to support more or less the same functionality. What's the difference, and when would you use one over the other?

Gamut

Answer 1

Queue.PriorityQueue is a thread-safe class, while the heapq module makes no thread-safety guarantees. From the Queue module documentation:

The Queue module implements multi-producer, multi-consumer queues. It is especially useful in threaded programming when information must be exchanged safely between multiple threads. The Queue class in this module implements all the required locking semantics. It depends on the availability of thread support in Python; see the threading module.

The heapq module offers no locking, and operates on standard list objects, which are not meant to be thread-safe.

In fact, the PriorityQueue implementation uses heapq under the hood to do all prioritisation work, with the base Queue class providing the locking to make this thread-safe. See the source code for details.

This makes the heapq module faster; there is no locking overhead. In addition, you are free to use the various heapq functions in different, novel ways, the PriorityQueue only offers the straight-up queueing functionality.

Regenerative

Answer 2

queue.PriorityQueue is a partial wrapper around the heapq class.

In other words, a queue.PriorityQueue is actually a heapq, placed in the queue module with a couple of renamed methods to make the heapq easier to use, much like a regular queue.

In heapq, you use use the method heappush() to add a new item and the method heappop() to remove one. That is not very queue-like, so queue.PriorityQueue let you use the usual queue methods such as put and get to do the same thing.

There are some features of heapq that are not carried over into queue.PriorityQueue, such as heappushpop() and heapreplace(), but you are less likely to use those. If you need them (and I do in my current project), perhaps you should use heapq rather than queue.PriorityQueue.

Also, since heapq is specialized for its purpose, it is not thread safe (as noted in another answer here.)

Postliminy

Answer 3

Yes, I have found the code. It is clear that PriorityQueue uses heapq.

class PriorityQueue(Queue):
    '''Variant of Queue that retrieves open entries in priority order (lowest first).

    Entries are typically tuples of the form:  (priority number, data).
    '''

    def _init(self, maxsize):
        self.queue = []

    def _qsize(self):
        return len(self.queue)

    def _put(self, item):
        heappush(self.queue, item)

    def _get(self):
        return heappop(self.queue)

Christoffer