How to paint / deform a QImage in 2D?

QPainter::transform() is indeed the solution as I already recommended in my comment. I was not fully sure about this but QPainter::transform() even covers drawn images deforming the original image rectangle respectively. (Otherwise, I had applied the transformation to the QImage itself.)

However, while making a small sample I realized that's only half of the story.

A small Qt sample application was done ASAP but I struggled to find a way to setup the transformation properly.

By the way I had to realize that translate(), scale(), and shear() are at best good for a 3 point deformation. A 4 point deformation may introduce perspective distortion as well. Hence, project() may be needed also. Feeling, that I had reached the border of my personal math capabilities I googled how other guys have solved that and found

OpenCV getPerspectiveTransform()

Calculates a perspective transform from four pairs of the corresponding points.

That sounded promising. Having a look into the (not so long) implementation of cv::getPerspectiveTransform(), I realized that they made a linear equation and used a solver to yield the respective transformation.

So, I did another search on google and found an IMHO straight-forward implementation by Martin Thoma Solving linear equations with Gaussian elimination. (I somehow remember that I must have heard about Gaussian elimination in my math lessons but that's decades ago, and I never have needed this in daily business since then.)

So, this is what I did for a solver (applying minor stylistic changes to the original code of Martin Thoma) – solveLin.h:

#ifndef SOLVE_LIN_H
#define SOLVE_LIN_H

#include <cassert>
#include <cmath>
#include <vector>

template <typename VALUE>
class MatrixT {
  public:
    typedef VALUE Value;

  private:
    std::vector<Value> _values;
    size_t _nCols;

  public:
    explicit MatrixT(
      size_t nRows, size_t nCols, const Value &value = Value()):
      _values(nRows * nCols, value),
      _nCols(nCols)
    { }
    explicit MatrixT(
      size_t nRows, size_t nCols,
      std::initializer_list<Value> values):
      _values(/*assert(values.size() == nRows * nCols),*/ values),
      _nCols(nCols)
    { }
    ~MatrixT() = default;
    MatrixT(const MatrixT&) = default;
    MatrixT& operator=(const MatrixT&) = default;

    size_t cols() const { return _nCols; }
    size_t rows() const { return _values.size() / _nCols; }
    const Value* operator[](size_t row) const
    {
      assert(row < rows());
      return &_values[row * _nCols];
    }
    Value* operator[](const size_t row)
    {
      return (Value*)((const MatrixT&)*this)[row];
    }
};

/** strongly inspired by (not to say: shamelessly copied from)
 * Martin Thoma "Solving linear equations with Gaussian elimination"
 * https://martin-thoma.com/solving-linear-equations-with-gaussian-elimination/
 */
template <typename VALUE>
std::vector<VALUE> gauss(MatrixT<VALUE> mat)
{
  typedef VALUE Value;
  const size_t n = mat.rows();
  assert(mat.cols() == n + 1);
  for (size_t i = 0; i < n; ++i) {
    // search for max. value in this column
    Value maxI = std::abs(mat[i][i]);
    size_t iMax = i;
    for (size_t k = i + 1; k < n; ++k) {
      const Value maxK = std::abs(mat[k][i]);
      if (maxI < maxK) maxI = maxK, iMax = k;
    }
    // swap max. row with current row
    std::swap_ranges(
      mat[i] + i, mat[i] + n + 1,
      mat[iMax] + i);
    // make all rows below this one 0 in current column
    for (size_t k = i + 1; k < n; ++k) {
      const Value c = mat[k][i] / mat[i][i];
      for (size_t j = i; j < n + 1; ++j) {
        if (i == j) mat[k][j] = (Value)0;
        else mat[k][j] -= c * mat[i][j];
      }
    }
  }
  // solve equation Ax=b for an upper triangular matrix A
  std::vector<Value> x(n);
  for (size_t i = n; i--;) {
    x[i] = mat[i][n] / mat[i][i];
    for (size_t k = i; k--;) {
      mat[k][n] -= mat[k][i] * x[i];
    }
  }
  // done
  return x;
}

#endif // SOLVE_LIN_H

The main application testQImage4Point.cc:

#include <QtWidgets>

#include "solveLin.h"

/* strongly inspired by (not to say: shamelessly copied from)
 * cv::Mat cv::getPerspectiveTransform(
 *   const Point2f src[], const Point2f dst[], int solveMethod)
 */ 
QTransform xform4Point(
  const QPoint quad0[4], const QPoint quad1[4])
{
  qDebug() << "quad0:" << quad0[0] << quad0[1] << quad0[2] << quad0[3];
  qDebug() << "quad1:" << quad1[0] << quad1[1] << quad1[2] << quad1[3];
  typedef MatrixT<double> Matrix;
  Matrix mat(8, 9, 0.0);
  for (size_t i = 0; i < 4; ++i) {
    mat[i][0] = mat[i + 4][3] = quad0[i].x();
    mat[i][1] = mat[i + 4][4] = quad0[i].y();
    mat[i][2] = mat[i + 4][5] = 1.0;
    mat[i][6] = -quad0[i].x() * quad1[i].x();
    mat[i][7] = -quad0[i].y() * quad1[i].x();
    mat[i + 4][6] = -quad0[i].x() * quad1[i].y();
    mat[i + 4][7] = -quad0[i].y() * quad1[i].y();
    mat[i][8] = quad1[i].x();
    mat[i + 4][8] = quad1[i].y();
  }
  std::vector<double> result = gauss(mat);
  return QTransform(
    result[0], result[3], result[6],
    result[1], result[4], result[7],
    result[2], result[5], 1.0);
}

class Canvas: public QWidget {
  private:
    QImage _qImg;
    QTransform _qXform;
    QPoint _quadOrig[4];
    QPoint _quadXform[4];
    int _editMode;
    bool _viewXform;
    QSize _gripSize;
    QPoint *_quadEdit; // pointer to currently edited quad
    int _grip; // current grip (in mouse drag)

  public:
    Canvas();
    virtual ~Canvas() = default;
    Canvas(const Canvas&) = delete;
    Canvas& operator=(const Canvas&) = delete;

  public:
    const QImage& image() const { return _qImg; }
    void setImage(const QImage &qImg);
    int editMode() const { return _editMode; }
    void setEditMode(int editMode);
    bool viewXform() const { return _viewXform; }
    void setViewXform(bool enable);

  protected:
    virtual void paintEvent(QPaintEvent *pQEvent) override;
    virtual void mousePressEvent(QMouseEvent *pQEvent) override;
    virtual void mouseMoveEvent(QMouseEvent *pQEvent) override;

  private:
    int pickGrip(const QPoint &pos) const;
    void drawQuad(QPainter &qPainter, const QPoint quad[4], bool grips);
    void drawGrid(QPainter &qPainter);
};

Canvas::Canvas():
  QWidget(),
  _quadOrig{
    QPoint(0.25 * width(), 0.25 * height()),
    QPoint(0.75 * width(), 0.25 * height()),
    QPoint(0.75 * width(), 0.75 * height()),
    QPoint(0.25 * width(), 0.75 * height())
  },
  _quadXform{
    _quadOrig[0], _quadOrig[1], _quadOrig[2], _quadOrig[3]
  },
  _editMode(0),
  _viewXform(true),
  _gripSize(7, 7),
  _quadEdit(_quadOrig),
  _grip(-1)
{ }

void Canvas::setImage(const QImage &qImg)
{
  _qImg = qImg;
  _quadOrig[0] = QPoint(0.25 * _qImg.width(), 0.25 * _qImg.height());
  _quadOrig[1] = QPoint(0.75 * _qImg.width(), 0.25 * _qImg.height());
  _quadOrig[2] = QPoint(0.75 * _qImg.width(), 0.75 * _qImg.height());
  _quadOrig[3] = QPoint(0.25 * _qImg.width(), 0.75 * _qImg.height());
  std::copy(_quadOrig, _quadOrig + 4, _quadXform);
  update();
}

void Canvas::setEditMode(int editMode)
{
  _editMode = editMode;
  _quadEdit
    = _editMode == 0 ? _quadOrig
    : _editMode == 1 ? _quadXform
    : nullptr;
  update();
}

void Canvas::setViewXform(bool enable)
{
  _viewXform = enable;
  update();
}

void Canvas::paintEvent(QPaintEvent *pQEvent)
{
  QWidget::paintEvent(pQEvent);
  QPainter qPainter(this);
  const QTransform qXformOld = qPainter.transform();
  if (_viewXform) qPainter.setTransform(_qXform);
  qPainter.drawImage(0, 0, _qImg);
  qPainter.setPen(Qt::white);
  drawGrid(qPainter);
  qPainter.setPen(Qt::black);
  qPainter.setPen(Qt::DotLine);
  drawGrid(qPainter);
  qPainter.setPen(Qt::SolidLine);
  qPainter.setTransform(qXformOld);
  for (int i = 1; i <= 2; ++i) {
    switch ((_editMode + i) % 2) {
      case 0: // draw orig. quad
        qPainter.setPen(Qt::red);
        drawQuad(qPainter, _quadOrig, _editMode == 0);
        break;
      case 1:
        // draw transformed quad
        qPainter.setPen(Qt::green);
        drawQuad(qPainter, _quadXform, _editMode == 1);
        break;
    }
  }
}

void Canvas::mousePressEvent(QMouseEvent *pQEvent)
{
  if (pQEvent->button() == Qt::LeftButton) {
    _grip = pickGrip(pQEvent->pos());
    qDebug() << "hit:" << _grip;
  }
}

void Canvas::mouseMoveEvent(QMouseEvent *pQEvent)
{
  if ((pQEvent->buttons() & Qt::LeftButton) && _grip >= 0) {
    _quadEdit[_grip] = pQEvent->pos();
    _qXform = xform4Point(_quadOrig, _quadXform);
    qDebug() << "transform:" << _qXform;
    update();
  }
}

int Canvas::pickGrip(const QPoint &pos) const
{
  if (!_quadEdit) return -1;
  const QPoint gripOffs(_gripSize.width() / 2, _gripSize.height() / 2);
  for (int i = 4; i--;) {
    const QRect rect(_quadEdit[i] - gripOffs, _gripSize);
    if (rect.contains(pos)) return i;
  }
  return -1;
}

void Canvas::drawQuad(QPainter &qPainter, const QPoint quad[4], bool grips)
{
  qPainter.drawPolygon(quad, 4);
  if (grips) {
    const QPoint gripOffs(_gripSize.width() / 2, _gripSize.height() / 2);
    for (int i = 0; i < 4; ++i) {
      qPainter.drawRect(QRect(quad[i] - gripOffs, _gripSize));
    }
  }
}

void Canvas::drawGrid(QPainter &qPainter)
{
  const int w = _qImg.width() - 1, h = _qImg.height() - 1;
  const int n = 5;
  for (int i = 0; i <= n; ++i) {
    const int x = i * w / n, y = i * h / n;
    qPainter.drawLine(x, 0, x, h);
    qPainter.drawLine(0, y, w, y);
  }
}

int main(int argc, char **argv)
{
  qDebug() << "Qt Version:" << QT_VERSION_STR;
  QApplication app(argc, argv);
  // init GUI
  QMainWindow winMain;
  winMain.setWindowTitle("4 Point Transform");
  Canvas canvas;
  winMain.setCentralWidget(&canvas);
  QToolBar qToolBar;
  QActionGroup qTglGrpEdit(&qToolBar);
  QAction qTglFrom("Edit From", &qTglGrpEdit);
  qTglFrom.setCheckable(true);
  if (canvas.editMode() == 0) qTglFrom.setChecked(true);
  qToolBar.addAction(&qTglFrom);
  QAction qTglTo("Edit To", &qTglGrpEdit);
  qTglTo.setCheckable(true);
  if (canvas.editMode() == 1) qTglTo.setChecked(true);
  qToolBar.addAction(&qTglTo);
  qToolBar.addSeparator();
  QActionGroup qTglGrpView(&qToolBar);
  QAction qTglOrig("View Original", &qTglGrpView);
  qTglOrig.setCheckable(true);
  if (!canvas.viewXform()) qTglOrig.setChecked(true);
  qToolBar.addAction(&qTglOrig);
  QAction qTglXform("View Deformed", &qTglGrpView);
  qTglXform.setCheckable(true);
  if (canvas.viewXform()) qTglXform.setChecked(true);
  qToolBar.addAction(&qTglXform);
  winMain.addToolBar(&qToolBar);
  winMain.show();
  // init image
  const int dx = winMain.width() - canvas.width();
  const int dy = winMain.height() - canvas.height();
  canvas.setImage(QImage("window-cats.jpg"));
  winMain.resize(canvas.image().width() + dx, canvas.image().height() + dy);
  // install signal handlers
  QObject::connect(&qTglFrom, &QAction::triggered,
    [&](bool checked) { if (checked) canvas.setEditMode(0); });
  QObject::connect(&qTglTo, &QAction::triggered,
    [&](bool checked) { if (checked) canvas.setEditMode(1); });
  QObject::connect(&qTglOrig, &QAction::triggered,
    [&](bool checked) { if (checked) canvas.setViewXform(false); });
  QObject::connect(&qTglXform, &QAction::triggered,
    [&](bool checked) { if (checked) canvas.setViewXform(true); });
  // runtime loop
  return app.exec();
}

I made a project file to compile in cygwin – testQImage4Point.pro:

SOURCES = testQImage4Point.cc

QT += widgets

which can be built and run with the following commands:

$ qmake-qt5 testQImage4Point.pro

$ make && ./testQImage4Point 
g++ -c -fno-keep-inline-dllexport -D_GNU_SOURCE -pipe -O2 -Wall -W -D_REENTRANT -DQT_NO_DEBUG -DQT_WIDGETS_LIB -DQT_GUI_LIB -DQT_CORE_LIB -I. -isystem /usr/include/qt5 -isystem /usr/include/qt5/QtWidgets -isystem /usr/include/qt5/QtGui -isystem /usr/include/qt5/QtCore -I. -I/usr/lib/qt5/mkspecs/cygwin-g++ -o testQImage4Point.o testQImage4Point.cc
g++  -o testQImage4Point.exe testQImage4Point.o   -lQt5Widgets -lQt5Gui -lQt5Core -lGL -lpthread 
Qt Version: 5.9.4

As I had to debug the code, I made a CMakeLists.txt as well:

project(QImage4Point)

cmake_minimum_required(VERSION 3.10.0)

set_property(GLOBAL PROPERTY USE_FOLDERS ON)
set(CMAKE_CXX_STANDARD 17)
set(CMAKE_CXX_STANDARD_REQUIRED ON)
set(CMAKE_CXX_EXTENSIONS OFF)

find_package(Qt5Widgets CONFIG REQUIRED)

include_directories(
  "${CMAKE_SOURCE_DIR}")

add_executable(testQImage4Point
  testQImage4Point.cc
  solveLin.h)

target_link_libraries(testQImage4Point
  Qt5::Widgets)

# define QT_NO_KEYWORDS to prevent confusion between of Qt signal-slots and
# other signal-slot APIs
target_compile_definitions(testQImage4Point PUBLIC QT_NO_KEYWORDS)

which I used to make a VS2017 solution.

This is how the code looked in action after I got it working as expected:

What I did not yet: applying the transformation to transform a QImage directly into another QImage. I think it's possible. For the resulting QImage, every pixel has to be looked up in the source applying the inverse transformation to the coordinates. Thereby, of course, transformed coordinates might be out of range. Hence, this case has to be handled (e.g. returning a pre-defined border color). (This is what I did in the answer to a similar question
SO: Rotate an image in C++ without using OpenCV functions
which incidentally came one day later.)

OpenCV `getPerspectiveTransform()`

Recommended topics

Hot tags

OpenCV getPerspectiveTransform()

Recommended topics

Hot tags

OpenCV `getPerspectiveTransform()`