GtkTreeView insert/update performance penalty because of sorting

Question

I'm having performance problems when inserting many rows into a GTK treeview (using PyGTK) - or when modifying many rows. The problem is that the model seems to get resorted after each change (insert/modification). This causes the GUI to hang for multiple seconds. Leaving the model unsorted by commenting out model.set_sort_column_id(SOME_ROW_INDEX, gtk.SORT_ASCENDING) eliminates these problems.

Therefore, I would like to disable the sorting while I'm inserting or modifying the model, and re-enable it afterwards. Unfortunately, sorting can't be disabled with model.set_sort_column_id(-1, gtk.SORT_ASCENDING). Using the freeze/thaw functions doesn't work either:

treeview.freeze_child_notify()

try:
    for row in model:
        # ... change something in row ...
finally:
    treeview.thaw_child_notify()

So, how can I disable the sorting? Or is there a better method for bulk inserts/modifications?

---

Solution

Thanks to the information and links bobince provided in his answer, I checked out some of the alternatives:

1) Dummy sorting

 tv.freeze_child_notify()
 sortSettings = model.get_sort_column_id()
 model.set_default_sort_func(lambda *unused: 0) # <-- can also use None but that is slower!
 # model.set_default_sort_func(lambda *unused: 1) <-- slow
 # model.set_default_sort_func(lambda *unused: -1) <-- crash (access violation in gtk_tree_store_move_after?!)
 model.set_sort_column_id(-1, gtk.SORT_ASCENDING)
 # change rows
 model.set_sort_column_id(*sortSettings)
 tv.thaw_child_notify()

This brought the time down from about 11 seconds to 2 seconds. Wow! But could be better, this was only for 1000 rows.

2) Removing model while updating

tv.set_model(None)
# change rows
tv.set_model(model)

No noticable difference, 11 seconds.

3) Dummy sorting and the cool generator trick from the PyGTK FAQ

 def gen():
      tv.freeze_child_notify()
      sortSettings = model.get_sort_column_id()
      model.set_default_sort_func(lambda *unused: 0)
      model.set_sort_column_id(-1, gtk.SORT_ASCENDING)

      i = 0
      for row in rowsToChange:
           i += 1
           # change something
           if i % 200 == 0:
                # freeze/thaw not really  necessary here as sorting is wrong because of the
                # default sort function
                yield True

      model.set_sort_column_id(*sortSettings)
      tv.thaw_child_notify()
      yield False

 g = gen()
 if g.next(): # run once now, remaining iterations when idle
     gobject.idle_add(g.next)

The result: The same estimated 2 seconds as in solution 1), but the GUI reacts during this time. I prefer this method. The modulo 200 can be tweaked to make the GUI more or less reactive if needed.

Maybe it's even possible to subclass gtk.TreeStore to get better results? Haven't tried that yet, but option 3) is good enough for now.

Nerveracking

Answer 1

Sounds like you're nearly there. See the FAQ for further notes. In particular, you should also set the default_sort_order (you can now use None as well as the dummy compare lambda in that example, for better performance) to ensure there is no sorting, and remove the model from the treeview for the duration of the operations.

If it's a lot of changes you may be better off creating and setting a complete new model.

Languorous

Thanks, that was really helpful :) See my question for some solutions I've tested based on the info you provided. – Nerveracking