3 node cluster, 1 HaProxy.
We had a few incidents yesterday, where node1 was crashed,
When restarted, nodes 2 AND 3 decided to crash.
Node 2 MariaDB/MySQL could not be restarted ("Signal 11" immediately after starting Recovery)
We ended up having to restore node 2, by removing MySQL and datafiles, reinstalling MySQL and restoring configfiles.
After restart, it did SST and started working again.
However, this rendered ClusterControl unable to see the node. (OS)
Galera cluster is working as it should, when checking manually in MySQL/MariaDB, only problem is that cmon lost track of this one node.
Nothing was changed regarding Linux user, all MySQL users are recreated as on node 1/3
cmon GUI can show everything DB related (Communityversion-wise)
But node keeps being "orange" in status "Disconnected/Down"
It got a little better after restarting cmon and apache2 on the clustercontrol linux server. But I am lost here now.
I hope someone here has a suggestion, that does not include taking down cluster or losing valuable performance history in cmon ;)
Please sign in to leave a comment.