Constant hanging of wsrep in pre-commit stage and Preparing for TO isolation
The galera cluster I am working on has no other users or processes running on it except me trying to restore a dump. Yet I am constantly running into these to hangs. We are at a point we are about to give up on galera clustering technologies due to all the errors that keep happening. I am trying to insert a 200MB table. Shouldn't be this much of an issue:
| 376276 | root | localhost | SBDBRMDEV01 | Query | 192 | Preparing for TO isolation | INSERT INTO `Table` VALUES (INT,'VARCHAR',INT,'0','VARCHAR',INT,INT,'DATE |
-
It's hard to tell what exactly is happening in your cluster but the 'Preparing for TO isolation' means that a cluster is waiting for at least one of the nodes to get in sync because a query has to be executed using TOI. Frankly, I'm not sure why it stuck on the INSERT query. Can you please provide following data:
SHOW FULL PROCESSLIST;
SHOW GLOBAL VARIABLES\G
SHOW STATUS LIKE 'wsrep%';
SHOW ENGINE InnoDB STATUS\G
Error logfrom all of the nodes. Additionally, can you please send us output of SHOW CREATE TABLE Table\G where the insert was taking place?
-
mysql> SHOW FULL PROCESSLIST;
+------+-----------------+-------------------+------------+---------+-------+-----------------------------+-----------------------+-----------+---------------+
| Id | User | Host | db | Command | Time | State | Info | Rows_sent | Rows_examined |
+------+-----------------+-------------------+------------+---------+-------+-----------------------------+-----------------------+-----------+---------------+
| 1 | system user | | NULL | Sleep | 16904 | NULL | NULL | 0 | 0 |
| 2 | system user | | NULL | Sleep | 16904 | wsrep aborter idle | NULL | 0 | 0 |
| 3 | event_scheduler | localhost | NULL | Daemon | 6278 | Waiting for next activation | NULL | 0 | 0 |
| 4 | system user | | NULL | Sleep | 6278 | NULL | NULL | 0 | 0 |
| 5 | system user | | NULL | Sleep | 6278 | NULL | NULL | 0 | 0 |
| 6 | system user | | NULL | Sleep | 6278 | NULL | NULL | 0 | 0 |
| 7 | system user | | NULL | Sleep | 6278 | NULL | NULL | 0 | 0 |
| 8 | system user | | NULL | Sleep | 6278 | NULL | NULL | 0 | 0 |
| 9 | system user | | NULL | Sleep | 6278 | NULL | NULL | 0 | 0 |
| 10 | system user | | NULL | Sleep | 6278 | NULL | NULL | 0 | 0 |
| 11 | cmon | 10.1.105.45:51499 | gtowebii | Sleep | 0 | | NULL | 0 | 0 |
| 12 | cmon | 10.1.105.45:51500 | phpmyadmin | Sleep | 0 | | NULL | 459 | 459 |
| 6401 | root | localhost | NULL | Query | 0 | init | SHOW FULL PROCESSLIST | 0 | 0 |
+------+-----------------+-------------------+------------+---------+-------+-----------------------------+-----------------------+-----------+---------------+
Please sign in to leave a comment.
Comments
12 comments