MySQL - max_binlog_cache_size vs binlog_cache_size

There is quite a lot of confusion in the description of these variables, in official documentation of MySQL.

According to it, max_binlog_cache_size means,

If a transaction requires more than this many bytes of memory, the server generates a Multi-statement transaction required more than 'max_binlog_cache_size' bytes of storage error.

max_binlog_cache_size sets the size for the transaction cache only

and binlog_cache_size means,

The size of the cache to hold changes to the binary log during a transaction.

binlog_cache_size sets the size for the transaction cache only

On reading the documentation, I observed there is no difference among these two. There is also something very confusing in the documentation like,

In MySQL 5.7, the visibility to sessions of max_binlog_cache_size matches that of the binlog_cache_size system variable; in other words, changing its value effects only new sessions that are started after the value is changed.

When I query the server variables it shows both. I have a MySQL 5.6 and a MySQL 5.7. All I need to know is, which variable I should consider and configure for which server.

binlog_cache_size for MySQL 5.6 and max_binlog_cache_size for MySQL 5.7??

There are additional confusing variables max_binlog_stmt_cache_size and binlog_stmt_cache_size, related to these.

Both variables can be configured in both versions, they have different meaning. Definitions in the manual and in the help are confusing; here is a much better explanation: http://dev.mysql.com/doc/refman/5.6/en/binary-log.html

binlog_cache_size defines the maximum amount of memory that the buffer can use. If transaction grows above this value, it uses a temporary disk file. Please note that the buffer is allocated per connection.

max_binlog_cache_size defines the maximum total size of a transaction. If the transaction grows above this value, it fails.

Below is a simple demonstration of the difference.

Setup:

MariaDB [test]> select @@binlog_cache_size, @@max_binlog_cache_size, @@binlog_format;
+---------------------+-------------------------+-----------------+
| @@binlog_cache_size | @@max_binlog_cache_size | @@binlog_format |
+---------------------+-------------------------+-----------------+
|               32768 |                   65536 | ROW             |
+---------------------+-------------------------+-----------------+
1 row in set (0.01 sec)

MariaDB [test]> show create table t1 \G
*************************** 1. row ***************************
       Table: t1
Create Table: CREATE TABLE `t1` (
  `a` text
) ENGINE=InnoDB DEFAULT CHARSET=latin1
1 row in set (0.00 sec)

1. Transaction size is below @@binlog_cache_size

(transaction succeeds, uses the cache, does not use the disk)

MariaDB [test]> flush status;
Query OK, 0 rows affected (0.00 sec)

MariaDB [test]> begin;
Query OK, 0 rows affected (0.00 sec)

MariaDB [test]> insert into t1 values (repeat('a',20000));
Query OK, 1 row affected (0.01 sec)

MariaDB [test]> insert into t1 values (repeat('a',10000));
Query OK, 1 row affected (0.04 sec)

MariaDB [test]> commit;
Query OK, 0 rows affected (0.05 sec)

MariaDB [test]> show status like 'Binlog_cache%';
+-----------------------+-------+
| Variable_name         | Value |
+-----------------------+-------+
| Binlog_cache_disk_use | 0     |
| Binlog_cache_use      | 1     |
+-----------------------+-------+
2 rows in set (0.01 sec)

2. Transaction size is above @@binlog_cache_size, but below @@max_binlog_cache_size

(transaction uses the cache, and the cache uses the disk)

MariaDB [test]> flush status;
Query OK, 0 rows affected (0.00 sec)

MariaDB [test]> begin;
Query OK, 0 rows affected (0.00 sec)

MariaDB [test]> insert into t1 values (repeat('a',20000));
Query OK, 1 row affected (0.10 sec)

MariaDB [test]> insert into t1 values (repeat('a',20000));
Query OK, 1 row affected (0.10 sec)

MariaDB [test]> commit;
Query OK, 0 rows affected (0.03 sec)

MariaDB [test]> show status like 'Binlog_cache%';
+-----------------------+-------+
| Variable_name         | Value |
+-----------------------+-------+
| Binlog_cache_disk_use | 1     |
| Binlog_cache_use      | 1     |
+-----------------------+-------+
2 rows in set (0.01 sec)

3. Transaction size exceeds @@max_binlog_cache_size

(transaction fails)

MariaDB [test]> flush status;
Query OK, 0 rows affected (0.00 sec)

MariaDB [test]> begin;
Query OK, 0 rows affected (0.00 sec)

MariaDB [test]> insert into t1 values (repeat('a',20000));
Query OK, 1 row affected (0.12 sec)

MariaDB [test]> insert into t1 values (repeat('a',20000));
Query OK, 1 row affected (0.15 sec)

MariaDB [test]> insert into t1 values (repeat('a',20000));
Query OK, 1 row affected (0.12 sec)

MariaDB [test]> insert into t1 values (repeat('a',20000));
ERROR 1197 (HY000): Multi-statement transaction required more than 'max_binlog_cache_size' bytes of storage; increase this mysqld variable and try again

So, if your transactions are big, but you don't have too many connections, you might want to increase @@binlog_cache_size to avoid excessive disk writes.

If you have many concurrent connections, you should be careful to avoid connections trying to allocate too much memory for the caches simultaneously.

If you want to make sure that transactions don't grow too big, you might want to limit @@max_binlog_cache_size.

@@binlog_stmt_cache_size and @@max_binlog_stmt_cache_size should work in a similar way, the difference is that %binlog_cache% values are for transactional updates, and %binlog_stmt_cache% for non-transactional updates.

While experimenting, please note that the values are not 100% precise, there are some hidden subtleties with initially allocated sizes. It shouldn't matter for practical purposes, but can be confusing when you play with low values.

Recommended topics

Hot tags