Non-blocking worker - interrupt file copy
Asked Answered
C

2

4

I'm dealing with very large files, in excess of hundreds of GB in size. The User needs to be able to move these files between disks and is on a restricted system with no default file manager. It's possible for the User to realize they've made a mistake and cancel the operation, and as far as I can tell the User will have to wait for the current copy or rename operation to complete. This can leave them feeling frustrated as they wait potentially minutes, only to see that their many GB file still copied. In the case of Copy I could delete the second file, but in the case of rename, which I'm using to move files, I'd have to repeat the operation in reverse to undo it, and that's simply not acceptable.

Is there some way to interrupt copy() and rename() that I'm not seeing in the documentation for QFile, or will I need to put together my own class to handle copy and rename?

Cilium answered 5/10, 2015 at 15:38 Comment(0)
G
6

I don't think the file size has any effect on how long a renaming will take.

For the copy - Qt offers nothing built in, you have to implement it yourself. The key gotcha here is that you will have to find some way to poll for a copy cancellation continuously. This means you cannot lock the main thread in order to be able to process events.

Whether you go for an extra thread in order to keep the main thread responsive, or decide to use the main thread - in both cases you will need to implement "fragmented" copying - one chunk at a time using a buffer, until the file is copied or copying is cancelled. You need this to be able to process user events and track copying progress.

I suggest you implement a QObject derived copy helper worker class which tracks file name, total size, buffer size, progress and clean up on cancellation. Then it is a matter of choice whether you will use it in the main thread or in a dedicated thread.

EDIT: Found it, but you better double check it, since it was done as an example and has not been thoroughly tested:

class CopyHelper : public QObject {
    Q_OBJECT
    Q_PROPERTY(qreal progress READ progress WRITE setProgress NOTIFY progressChanged)
public:
    CopyHelper(QString sPath, QString dPath, quint64 bSize = 1024 * 1024) :
        isCancelled(false), bufferSize(bSize), prog(0.0), source(sPath), destination(dPath), position(0) { }
    ~CopyHelper() { free(buff); }

    qreal progress() const { return prog; }
    void setProgress(qreal p) {
        if (p != prog) {
            prog = p;
            emit progressChanged();
        }
    }

public slots:
    void begin() {
        if (!source.open(QIODevice::ReadOnly)) {
            qDebug() << "could not open source, aborting";
            emit done();
            return;
        }
        fileSize = source.size();
        if (!destination.open(QIODevice::WriteOnly)) {
            qDebug() << "could not open destination, aborting";
            // maybe check for overwriting and ask to proceed
            emit done();
            return;
        }
        if (!destination.resize(fileSize)) {
            qDebug() << "could not resize, aborting";
            emit done();
            return;
        }
        buff = (char*)malloc(bufferSize);
        if (!buff) {
            qDebug() << "could not allocate buffer, aborting";
            emit done();
            return;
        }
        QMetaObject::invokeMethod(this, "step", Qt::QueuedConnection);
        //timer.start();
    }
    void step() {
        if (!isCancelled) {
            if (position < fileSize) {
                quint64 chunk = fileSize - position;
                quint64 l = chunk > bufferSize ? bufferSize : chunk;
                source.read(buff, l);
                destination.write(buff, l);
                position += l;
                source.seek(position);
                destination.seek(position);
                setProgress((qreal)position / fileSize);
                //std::this_thread::sleep_for(std::chrono::milliseconds(100)); // for testing
                QMetaObject::invokeMethod(this, "step", Qt::QueuedConnection);
            } else {
                //qDebug() << timer.elapsed();
                emit done();
                return;
            }
        } else {
            if (!destination.remove()) qDebug() << "delete failed";
            emit done();
        }
    }
    void cancel() { isCancelled = true; }

signals:
    void progressChanged();
    void done();

private:
    bool isCancelled;
    quint64 bufferSize;
    qreal prog;
    QFile source, destination;
    quint64 fileSize, position;
    char * buff;
    //QElapsedTimer timer;
};

The done() signal is used to deleteLater() the copy helper / close copy dialog or whatever. You can enable the elapsed timer and use it to implement an elapsed time property and estimated time as well. Pausing is another possible feature to implement. Using QMetaObject::invokeMethod() allows the event loop to periodically process user events so you can cancel and update progress, which goes from 0 to 1. You can easily tweak it for moving files as well.

Gundry answered 5/10, 2015 at 16:4 Comment(6)
I was afraid of that. Just a case of hours I'd rather be spent elsewhere. I'll do something with the pattern you suggested, and it will have to be in another thread to ensure the GUI can update with progress. On the bright side, I'll be able to give file specific progress this way.Cilium
As for rename(), I'm using it to move files. When you move to another directory on the same disk it's constant, but going from one disk to another does take time depending on the file size.Cilium
@Cilium - I think I have done something like this already, will check when I get back home in a couple of hours.Gundry
This is what I ended up doing in order to keep the UI responsive.Dichloride
@Cilium - found it, will probably save you some time.Gundry
@ddriver That will definitely give me a boost, thanks for the help.Cilium
L
2

I don't think the function you are looking for exists.

What you can do is instead of using the copy() function, create a new file and gradualy read(qint64 maxSize) to a QByteArray from the old file and write(const QByteArray & byteArray) to the new file. This way you can control the flow yourself, just check if the user hasn't pressed cancel between every read/write.

Lutestring answered 5/10, 2015 at 15:55 Comment(0)

© 2022 - 2024 — McMap. All rights reserved.