deceptively simple implementation of topological sorting in python
Asked Answered
S

5

32

Extracted from here we got a minimal iterative dfs routine, i call it minimal because you can hardly simplify the code further:

def iterative_dfs(graph, start, path=[]):
    q = [start]
    while q:
        v = q.pop(0)
        if v not in path:
            path = path + [v]
            q = graph[v] + q

    return path

graph = {
    'a': ['b', 'c'],
    'b': ['d'],
    'c': ['d'],
    'd': ['e'],
    'e': []
}
print(iterative_dfs(graph, 'a'))

Here's my question, how could you transform this routine into a topological sort method where the routine also becomes "minimal"? I've watched this video and the idea is quite clever so I was wondering if it'd be possible to apply the same trick into the above code so the final result of topological_sort also becomes "minimal".

Not asking for a version of topological sorting which is not a tiny modification of the above routine, i've already seen few of them. The question is not "how do i implement topological sorting in python" but instead, finding the smallest possible set of tweaks of the above code to become a topological_sort.

ADDITIONAL COMMENTS

In the original article the author says :

A while ago, I read a graph implementation by Guido van Rossen that was deceptively simple. Now, I insist on a pure python minimal system with the least complexity. The idea is to be able to explore the algorithm. Later, you can refine and optimize the code but you will probably want to do this in a compiled language.

The goal of this question is not optimizing iterative_dfs but instead coming up with a minimal version of topological_sort derived from it (just for the sake of learning more about graph theory algorithms). In fact, i guess a more general question could be something like given the set of minimal algorithms, {iterative_dfs, recursive_dfs, iterative_bfs, recursive_dfs}, what would be their topological_sort derivations? Although that would make the question more long/complex, so figuring out the topological_sort out of iterative_dfs is good enough.

Saritasarkaria answered 9/11, 2017 at 1:51 Comment(10)
@TomKarzes Understand now your meaning, that makes total sense, good idea! Do you think it'd be too "expensive" to preprocess the graph converting the deps list to a set before traversal so you don't need to keep both structures in the graph. Let's say the type of graphs the app would be handling could be the cartesian product of something like {graph_many_nodes}x{all_nodes_have_many_deps, few_nodes_many_deps} so it's worth optimizing. Anyway, the question is more about minimal code (simpler as possible), so whether the solution it's fast or not would be irrelevant in this case.Saritasarkaria
also path=[]... just don't do it...Ibert
@Ibert You mean keep it as a local variable instead of being a parameter? Take a look to the original article i've put in the thread, if you think about it, the reason why the author has chosen having path=[] as a parameter is so the interface in both iterative and recursive versions can be the same, if you have it as a local var you'll fail this point.Saritasarkaria
@Saritasarkaria - sorry should have been clearer, path=[] keeps the values of [] between runs - which may be surprising to many... just run your code twice with the same input and watch in horror as it produces different results!Ibert
@Ibert Sorry, I don't understand, I've already tried to run the code multiple times here (multiple python processes). Already tried using a for loop to execute the algorithm with the same inputs and I always get the same results (it looks deterministic code to me), so, could you clarify how did you those different results you're talking about so i can reproduce it locally? ThanksSaritasarkaria
@Saritasarkaria - in this case it does work as intended but only because another technicality. the line with path = ... assigns new list to local var path and returns that, but if someone in the future went in and changed that to say path.append(v) which does 'the same thing' you will suddenly get path that keeps the values between calls - generally you don't want to put mutables as default args in pythonIbert
@Ibert Alright, i understand now your point, I kind of agree till certain extent... As far as the method postcondition be that a particular parameter remains the same after the final execution of the function call the algorithm remains correct. Now, we could of course discussing whether or not it's a good practice using default parameters as an alternative to use local variables, in that case I do agree, that's not a good practice as you're adding irrelevant implementation details to the public interface.Saritasarkaria
Hehe, it seems to me you guys are focusing too much on refactoring/fixing the "unoptimal" educative above code (we all know that particular code is just educative code, not production-code) instead on thinking about the solution to the given challenge ;)Saritasarkaria
@BPL, this was my first time answering bounty question and I was naturally excited about it.:(. Anyway, please don't think I'm demanding explanation or something when I ask , what made the accepted answer more acceptable to you? because actual sort function was simpler? accompanying text? answer's experience? more people thought it was better(upvotes)? I'm relatively new to this SO thing, and trying to get a feel for how this place actually works.ThanksSupercilious
@ShihabShahriar I understand why you're asking... To be honest I've struggled quite a lot deciding which answer to accept, both of them were really suitable to me. The reason why I've accepted Blckknght were mainly because the number of upvotes, it was the first one to be published, it can be more useful to the general public because it provides really good insights. On the other hand, if we strictly stick to the question i've formulated I think your answer is more suitable one... so... I dunno, maybe I was wrong picking up one Both answers are really nice but I needed to choose one....Saritasarkaria
H
46

It's not easy to turn an iterative implementation of DFS into Topological sort, since the change that needs to be done is more natural with a recursive implementation. But you can still do it, it just requires that you implement your own stack.

First off, here's a slightly improved version of your code (it's much more efficient and not much more complicated):

def iterative_dfs_improved(graph, start):
    seen = set()  # efficient set to look up nodes in
    path = []     # there was no good reason for this to be an argument in your code
    q = [start]
    while q:
        v = q.pop()   # no reason not to pop from the end, where it's fast
        if v not in seen:
            seen.add(v)
            path.append(v)
            q.extend(graph[v]) # this will add the nodes in a slightly different order
                               # if you want the same order, use reversed(graph[v])

    return path

Here's how I'd modify that code to do a topological sort:

def iterative_topological_sort(graph, start):
    seen = set()
    stack = []    # path variable is gone, stack and order are new
    order = []    # order will be in reverse order at first
    q = [start]
    while q:
        v = q.pop()
        if v not in seen:
            seen.add(v) # no need to append to path any more
            q.extend(graph[v])

            while stack and v not in graph[stack[-1]]: # new stuff here!
                order.append(stack.pop())
            stack.append(v)

    return stack + order[::-1]   # new return value!

The part I commented with "new stuff here" is the part that figures out the order as you move up the stack. It checks if the new node that's been found is a child of the previous node (which is on the top of the stack). If not, it pops the top of the stack and adds the value to order. While we're doing the DFS, order will be in reverse topological order, starting from the last values. We reverse it at the end of the function, and concatenate it with the remaining values on the stack (which conveniently are already in the correct order).

Because this code needs to check v not in graph[stack[-1]] a bunch of times, it will be much more efficient if the values in the graph dictionary are sets, rather than lists. A graph usually doesn't care about the order its edges are saved in, so making such a change shouldn't cause problems with most other algorithms, though code that produces or updates the graph might need fixing. If you ever intend to extend your graph code to support weighted graphs, you'll probably end up changing the lists to dictionaries mapping from node to weight anyway, and that would work just as well for this code (dictionary lookups are O(1) just like set lookups). Alternatively, we could build the sets we need ourselves, if graph can't be modified directly.

For reference, here's a recursive version of DFS, and a modification of it to do a topological sort. The modification needed is very small indeed:

def recursive_dfs(graph, node):
    result = []
    seen = set()

    def recursive_helper(node):
        for neighbor in graph[node]:
            if neighbor not in seen:
                result.append(neighbor)     # this line will be replaced below
                seen.add(neighbor)
                recursive_helper(neighbor)

    recursive_helper(node)
    return result

def recursive_topological_sort(graph, node):
    result = []
    seen = set()

    def recursive_helper(node):
        for neighbor in graph[node]:
            if neighbor not in seen:
                seen.add(neighbor)
                recursive_helper(neighbor)
        result.insert(0, node)              # this line replaces the result.append line

    recursive_helper(node)
    return result

That's it! One line gets removed and a similar one gets added at a different location. If you care about performance, you should probably do result.append in the second helper function too, and do return result[::-1] in the top level recursive_topological_sort function. But using insert(0, ...) is a more minimal change.

Its also worth noting that if you want a topological order of the whole graph, you shouldn't need to specify a starting node. Indeed, there may not be a single node that lets you traverse the entire graph, so you may need to do several traversals to get to everything. An easy way to make that happen in the iterative topological sort is to initialize q to list(graph) (a list of all the graph's keys) instead of a list with only a single starting node. For the recursive version, replace the call to recursive_helper(node) with a loop that calls the helper function on every node in the graph if it's not yet in seen.

Herson answered 11/11, 2017 at 3:22 Comment(3)
In the iterative implementation, variable q is actually a stack.Tubule
@Tubule Yes, that's true. The q variable in the original question's code is also a stack, albeit a much less efficient one since it pushes and pops from the start, rather than the end where lists have efficient operations. That's what makes the DFS a DFS. A regular (LIFO) queue would give a breadth-first traversal instead. That FIFO queue is not what is notable about the code we're discussing though. The stack named stack in the topological sorting code replaces the call stack of the recursive version.Herson
In the recursive DFS implementation, result = [] should be result = [node]Zeebrugge
S
11

My idea is based on two key observations:

  1. Don't pop the next item from stack, keep it to emulate stack unwinding.
  2. Instead of pushing all children to stack, just push one.

Both of these help us to traverse the graph exactly like recursive dfs. As the other answer here noted, this is important for this particular problem. The rest should be easy.

def iterative_topological_sort(graph, start,path=set()):
    q = [start]
    ans = []
    while q:
        v = q[-1]                   #item 1,just access, don't pop
        path = path.union({v})  
        children = [x for x in graph[v] if x not in path]    
        if not children:              #no child or all of them already visited
            ans = [v]+ans 
            q.pop()
        else: q.append(children[0])   #item 2, push just one child

    return ans

q here is our stack. In the main loop, we 'access' our current node v from the stack. 'access', not 'pop', because we need to be able to come back to this node again. We find out all unvisited children of our current node. and push only the first one to stack (q.append(children[0])), not all of them together. Again, this is precisely what we do with recursive dfs.

If no eligible child is found (if not children), we have visited the entire subtree under it. So it's ready to be pushed into ans. And this is when we really pop it.

(Of course, this is far from optimal- performance-wise. There are several rather simple ways performance could be improved, but I ignore these to keep things simple.)

Supercilious answered 11/11, 2017 at 7:25 Comment(3)
this is a great approach! conceptually very close to "pure" dfs!Skintight
@StevenPenny, I disagree. "topological ordering of a directed graph is a linear ordering of its vertices such that for every directed edge uv from vertex u to vertex v, u comes before v in the ordering." wiki linkSupercilious
in given example, my solution outputs: ['a', 'c', 'b', 'd', 'e'], which I think is the desired ordering, and is the ordering of the accepted solution too.Supercilious
B
4

I'm pretty new to this, but shouldn't topological sort based on DFS work no matter where in the graph you start? The current solutions (as of this writing) only traverse the entire graph for particular starting points in the example graph. (Although I haven't completely thought it through, it seems the problem occurs when hitting a vertex that has no neighbors to visit. If the algorithm hits such a node before traversing all the other vertices in the graph, then the results are truncated.)

Although it is not as simple as the OP would probably like, the following is an iterative topological sort using DFS that works regardless of the order of vertices explored.

```
from collections import deque

def iterative_top_sort(graph):
    result = deque() #using deque because we want to append left
    visited = set()

    #the first entry to the stack is a list of the vertices in the 
    #graph. 

    stack = [[key for key in graph]] #we want the stack to hold lists

    while stack:
      for i in stack[-1]: 
        if i in visited and i not in result: 
          result.appendleft(i)
        if i not in visited:
          visited.add(i)
          #add the vertex's neighbors to the stack
          stack.append(graph[i]) 
          break
      else: 
        stack.pop() 

    return result
```
Broadcasting answered 1/12, 2021 at 22:58 Comment(0)
H
1

I was also trying to simplify this so I came up with this:

from collections import deque

def dfs(graph, source, stack, visited):
    visited.add(source)

    for neighbour in graph[source]:
        if neighbour not in visited:
            dfs(graph, neighbour, stack, visited)
    
    stack.appendleft(source)

def topological_sort_of(graph):
    stack = deque()
    visited = set()

    for vertex in graph.keys():
        if vertex not in visited:
            dfs(graph, vertex, stack, visited)

    return stack

if __name__ == "__main__":
    graph = {
        0: [1, 2],
        1: [2, 5],
        2: [3],
        3: [],
        4: [],
        5: [3, 4],
        6: [1, 5],
    }

    topological_sort = topological_sort_of(graph)
    print(topological_sort)

Function dfs (Depth First Search) is used to create the stack of finishing times for every vertex in the graph. Finishing time here means that the element pushed into the stack first, is the first vertex where all of its neighbours are fully explored (no other unvisited neighbours are available to explore from that vertex) and the last element pushed into the stack is the last vertex where all of its neighbours are fully explored.

The stack is now simply the topological sort.

Using a Python set for visited provides constant membership checking and using deque as a stack provides constant-time left insertion as well.

The high-level idea was inspired by CLRS [1].

[1] Cormen, Thomas H., et al. Introduction to algorithms. MIT Press, 2009.

Homophonic answered 22/10, 2020 at 10:1 Comment(0)
M
1

Given your example graph:

a -->-- b -->-- d -->-- e
 \             /
  -->-- c -->--

We need to implement the graph, which you have done using "parent to children":

graph = {
   'a': ['b', 'c'],
   'b': ['d'],
   'c': ['d'],
   'd': ['e'],
   'e': [],
}

but you also provide a start parameter. In the context of a topological sort, if you provide a, then all is well:

[a, b, c, d, e]

but what if you provide b? All the implementations on this page currently return this:

[b, d, e]

which is not correct, as c would need to be done before d. To solve this, we can instead map "child to parents" [1][2]. Then instead of choosing a start we can choose an end:

def tsort(graph, end):
   b = set()
   l = []
   s = [end]
   while s:
      n = s[-1]
      b.add(n)
      for m in graph[n]:
         if not m in b:
            s.append(m)
      if s[-1] == n:
         s.pop()
         l.append(n)
   return l

graph = {
   'a': [],
   'b': ['a'],
   'c': ['a'],
   'd': ['b', 'c'],
   'e': ['d'],
}

print(tsort(graph, 'e')) # ['a', 'c', 'b', 'd', 'e']
  1. https://rosettacode.org/wiki/Topological_sort
  2. https://github.com/adonovan/gopl.io/blob/master/ch5/toposort/main.go
Montagu answered 27/4, 2021 at 23:29 Comment(0)

© 2022 - 2024 — McMap. All rights reserved.