This article to describes how to add an existing cluster into ClusterControl.
Supported clusters are:
- Galera Clusters
- Collection of MySQL Servers
- Mongodb/Tokumx
Assumptions:
- Dedicated server for CMON Controller and UI. Do not colocate with Cluster
- Use the same OS for the Controller/UI server as for the Cluster. Do not mix. It is un-tested.
- We assume in this guide that you install as root. Read more on server requirements.
Outline of steps:
- Setup SSH to the cluster nodes
- Preparations of Monitored MySQL Servers (part of the cluster)
- Installation
- Add the existing cluster
Limitations:
- All nodes must be started and able to connect to. For Galera, all nodes must be SYNCED, when adding the existing cluster
- If controller is deployed on an IP address (hostname=<ipaddr> in /etc/cmon.cnf) and MySQL/Galera server nodes are deployed without skip_name_resolve then GRANTs may fail. http://54.248.83.112/bugzilla/show_bug.cgi?id=141
- GALERA: If wsrep_cluster_address contains a garbd, then the installation may fail.
- GALERA: Auto-detection of the Galera nodes is based on the wsrep_cluster_address. Atleast one Galera node must have wsrep_cluster_address=S1,..,Sn set, where S1 to Sn denotes the Galera nodes part of the Galera cluster.
Setup SSH from Controller -> Cluster, and Controller -> Controller
On the controller do, as user 'root':
ssh-keygen -trsa
(press enter on all questions)
For each cluster node do:
ssh-copy-id root@<cluster_node_ip>
On the controller do (controller needs to be able to ssh to itself):
ssh-copy-id root@<controller_ip>
SSH as a Non-root user
For now you need to setup passwordless sudo.
See Read more on server requirements.
Monitored MySQL Servers - GRANTs
This section applies to "Galera clusters" and "Collection of MySQL Servers".
The mysql 'root' user on the monitored MySQL servers must have the WITH GRANT OPTION specified.
On the monitored mysql servers you need to be able to connect like this
mysql -uroot -p<root password> -h127.0.0.1 -P3306 #change port from 3306 to the MySQL port you use.
mysql -root -p<root password> -P3306 #change port from 3306 to the MySQL port you use.
and NOT LIKE THIS:
mysql -uroot <--- INVALID EXAMPLE
If you cannot connect to the mysql server(s) like the first example above you need to do:
mysql> GRANT ALL ON *.* TO 'root'@'127.0.0.1' IDENTIFIED BY '<root password>' WITH GRANT OPTION;
mysql> GRANT ALL ON *.* TO 'root'@'localhost' IDENTIFIED BY '<root password>' WITH GRANT OPTION;
Installation
Follow the instructions below. It will install a MySQL Server as well (if none is installed already) to host the monitoring and management data, the UI, and the CMON Controller with a minimal configuration.
$ wget http://severalnines.com/downloads/cmon/install-cc.sh
$ chmod +x install-cc.sh
$ sudo ./install-cc.sh
A longer version of the instructions are located here: http://support.severalnines.com/entries/23737976-ClusterControl-Installation
Add Existing Cluster
In the UI, press the button Add Existing Cluster.
- All cluster nodes must have the same OS
- All cluster nodes must be installed in the same place
- All cluster nodes must listen on the same mysql port
In the screenshot below you see an example, adding a Galera Cluster with one node running on 10.0.1.10 (the rest of the galera nodes are auto-detected).
The Galera node on 10.0.1.10 must have the wsrep_cluster_address set, and to verify that you can do (open a terminal on 10.0.1.10 first):
mysql -uroot -h127.0.0.1 -p
show global variables like 'wsrep_cluster_address';
mysql> show global variables like 'wsrep_cluster_address';
+-----------------------+------------------------------------------------------------+
| Variable_name | Value |
+-----------------------+------------------------------------------------------------+
| wsrep_cluster_address | gcomm://10.0.1.10,10.0.1.11,10.0.1.12 |
+-----------------------+------------------------------------------------------------+
1 row in set (0.01 sec)
Click "Add Cluster" and after a while the Cluster will appear, or else a "Job failed" will be shown on the screen and you can investigate what the problem is.
Comments
70 comments
Hi Mmin,
In the Job output you sent everything looks fine. Are there any messages coming after 37?
Best regards,
Art
No, unfortunately not.
And there's no cluster visible at url: https://localhost/clustercontrol/admin/#g:clusters...
Hi Mmin,
I'll create a ticket for this as it requires us to have a look at configuration files and other log files of your ClusterControl instance.
Let's continue there.
Best regards,
Art
sure, thanks a lot.
Just completed the first troubleshooting session. It turns out that cmon detects the OS by catching first the entries defined in: /etc/system-release and in case system-release doesn't exist by looking for /etc/redhat-release. Since all the server run on Oracle Linux, cmon can't detect the OS properly as /etc/system-release doesn't contain any string containing "Redhat". Art got development releasing a new version 1.2.12.1117 which uses first /etc/redhat-release to discover the OS. This patch resolves this issue and the "Add external cluster" completes just fine.
Once the job is completed, the cluster needs to be synchronized first. This is done by going into the clustercontroler web gui. Navigate to. "Settings" - "Cluster Registrations". A new windows appears and there click on: "Synchronize Clusters" (it's on the top, right-hand corner - opposite the title: Cluster Registration).
did a fresh install after having made some many changes while on the call with Art. Discovered while installing the cluster control packages that cmon is already updated to version 1.2.12.1117 in the repo. So no need to manually synchronise the cluster as with the latest packages the entire installation and the add existing cluster works just fine.
Thanks a lot Art and of course the development team in the background. Excellent work!
Hello there,
I am trying to add a cluster and I am getting the error "System failed to initiate a job command. You may want to try again.
[object Object]" - no errors in cmon.log. Can you let me know how I can debug this? I can ssh to cc itself, and all the servers.
Hi Brandon,
It looks like you have an issue with the communication between the UI and the cmon rpc.
Did you recently upgrade clustercontrol? If so, can you try if it works with another browser? Sometimes a Javascript get cached aggressively by a browser and a newer version isn't picked up.
Let me know if this works.
Best regards,
Art
Hey Art,
Thanks for the info; what happened was I installed CC, upgraded it after a few days, and the error happened when I tried to add the cluster. I resolved it by redeploying the server from scratch, so was uanble to test your recommendation. Nonetheless, thanks for your help.
Hi Johan,
I have deleted all nodes of a cluster. Thereafter, the cluster repaired.
Then, the cluster "Add Existing Cluster" reinserted.
Now I have the cluster in the Cluster Control 2 times indoors.
Where can I delete the old one?
Regards
Andreas
Please sign in to leave a comment.