PyQt QTableView prohibitively slow when scrolling with large data sets

Question

I have a program that loads a profile from a csv file and displays the data in a table. The loading of a pandas data frame to the table is fast because I used a custom model implementation of QAbstractTableModel, but the resizing of the QTableView widget is incredibly slow.

What can I do to make the resizing and scrolling smoother?

Florrie

show what you tried!!! we cannot guess the problem – Parsaye

Is it not clear? I load 10k lines in a table and the scrolling takes forever, what code do you need, the code to set a model to a table? In other languages there are enables for rendering control, but I have no idea of how to do that in the QT environment. – Sinnard

Similar question in C++: QTableView slow performance with 1000s of visible cells – Venomous

Using a QTreeView is even worse, also with the uniformRowHeight trick. – Sinnard

Answer 1

Well, I ended up modifying the custom table model I made to use numpy, and now it is blazing fast.

Updated 22-02-2020 Works as of Pandas 1.0.1:

Use this table model:

import numpy as np

class PandasModel(QtCore.QAbstractTableModel):
    """
    Class to populate a table view with a pandas dataframe
    """
    def __init__(self, data, parent=None):
        QtCore.QAbstractTableModel.__init__(self, parent)
        self._data = np.array(data.values)
        self._cols = data.columns
        self.r, self.c = np.shape(self._data)

    def rowCount(self, parent=None):
        return self.r

    def columnCount(self, parent=None):
        return self.c

    def data(self, index, role=QtCore.Qt.DisplayRole):
        if index.isValid():
            if role == QtCore.Qt.DisplayRole:
                return str(self._data[index.row(),index.column()])
        return None


    def headerData(self, p_int, orientation, role):
        if role == QtCore.Qt.DisplayRole:
            if orientation == QtCore.Qt.Horizontal:
                return self._cols[p_int]
            elif orientation == QtCore.Qt.Vertical:
                return p_int
        return None

Florrie

return self._data[index.row(),index.column()] was not working at all for me. From scipy.org a.item(*args) is very similar to a[args], except, instead of an array scalar, a standard Python scalar is returned. This can be useful for speeding up access to elements of the array and doing arithmetic on elements of the array using Python’s optimized math. Therefore I used row_column = tuple([index.row(),index.column()]) return self._data.item(row_column) – Acculturation