My own storage engine crashes because of a too small sort_buffer

Question

I am working on my own storage engine for MySQL. So far this storage engine works reliable and correct - but only for small (~100 MB) tables... For big tables, I get a segmentation fault, when I try to execute a query with an order by, so something like this will lead to a segfault:

select * from item order by i_author;

So I compiled MySQL in debug mode, and saw, that the there is now an assertion failure in the merge_buffers function in filesort.cc:

/* The following will fire if there is not enough space in sort_buffer */
DBUG_ASSERT(maxcount!=0);

Honestly I have no idea what I can change in my storage engine to make this error disappear. It first looked like I have to change the configuration paramater sort_buffer_size - but even setting this thing higher than the size of the table does change anything with this error.

Does anyone, who know how to write MySQL storage engines have any idea how to solve this?

Hugh

Answer 1

There is a similar issue reported with the falcon storage engine. The comment in the bug was,

It seems that the problem occurs in filesort when you have row estimates that are seriously wrong.

Most likely there is a bug in your storage engine somewhere, but that is a hard thing to debug through stack overflow.

Salt

Ah thanks a lot, this hint really solved the problem. According to the documentation, giving a correct estimation for the table size is not mandatory - obviously the documentation is wrong. What I did now is to record the high of my B+-Tree from my binary key index and calculate an upper bound for the size from that (since only an estimation is needed, giving the correct value would be too expensive). Now all my tests run on big tables. Thanks a lot for that tip! – Hugh