New MySQL Galera Cluster, the job always fails
Hi,
I have installed ClusterControl on Ubuntu 18.4 (via offline method) successfully. Now I am trying to deploy a new MySQL Galera cluster with 2 node (1 CC node and 2 DB node, all running ubuntu 18.4). I am following Caleb Curry's youtube videos tuts to deploy this cluster. Here is a list of what i did so far
- installed cluster control on 1 server
- setup passwordless SSH on all 3 servers
- click deploy new cluster, selected MySQL Galera and fill the forms as shown in following screenshots
- clicked deploy, the job starts and after sometime it fails (check job output at the bottom)
The DB servers are fresh/clean VMs, no packages have been installed and firewall (UFW) is disabled on all 3 server (CC, DB1 and DB2)
Form Step 1:
Form Step 2:
Job Output:
[14:47:21]:<em style='color: #c66211;'>172.16.101.179</em>:3306: Failed to setup MySql server.
[14:47:21]:<em style='color: #c66211;'>172.16.101.179</em>: Setup server failure, see the previous messages.
[14:47:21]:<em style='color: #c66211;'>172.16.101.179</em>:3306: failed remote (172.16.101.179) retval: 100 command: DEBIAN_FRONTEND=noninteractive LC_ALL=C apt-get -o Dpkg::Options::='--force-confnew' -o Dpkg::Options::='--force-confmiss' -qq -y --force-yes install mariadb-server mariadb-client mariadb-common mariadb-backup percona-toolkit 2>&1 output: W: --force-yes is deprecated, use one of the options starting with --allow instead.
E: Unable to correct problems, you have held broken packages.
[14:47:21]: output: W: --force-yes is deprecated, use one of the options starting with --allow instead.
E: Unable to correct problems, you have held broken packages.
[14:47:21]: command: DEBIAN_FRONTEND=noninteractive LC_ALL=C apt-get -o Dpkg::Options::='--force-confnew' -o Dpkg::Options::='--force-confmiss' -qq -y --force-yes install mariadb-server mariadb-client mariadb-common mariadb-backup percona-toolkit 2>&1
[14:47:21]:Failed to install package.
[14:47:20]:<em style='color: #c66211;'>172.16.101.179</em>: Installing <strong style='color: #110679;'>mariadb-server mariadb-client mariadb-common mariadb-backup percona-toolkit </strong>.
[14:47:20]: output: W: --force-yes is deprecated, use one of the options starting with --allow instead.
E: Unable to correct problems, you have held broken packages.
[14:47:20]: command: DEBIAN_FRONTEND=noninteractive LC_ALL=C apt-get -o Dpkg::Options::='--force-confnew' -o Dpkg::Options::='--force-confmiss' -qq -y --force-yes install mariadb-server mariadb-client mariadb-common mariadb-backup percona-toolkit 2>&1
[14:47:20]:Failed to install package.
[14:47:20]:<em style='color: #c66211;'>172.16.101.179</em>: Installing <strong style='color: #110679;'>mariadb-server mariadb-client mariadb-common mariadb-backup percona-toolkit </strong>.
[14:47:20]:<em style='color: #c66211;'>172.16.101.179</em>: Installing <strong style='color: #110679;'>percona-toolkit</strong>.
[14:47:20]:<em style='color: #c66211;'>172.16.101.179</em>: Installing <strong style='color: #110679;'>mariadb-backup</strong>.
[14:47:20]:<em style='color: #c66211;'>172.16.101.179</em>: Installing <strong style='color: #110679;'>mariadb-common</strong>.
[14:47:20]:<em style='color: #c66211;'>172.16.101.179</em>: Installing <strong style='color: #110679;'>mariadb-client</strong>.
[14:47:20]:<em style='color: #c66211;'>172.16.101.179</em>: Installing <strong style='color: #110679;'>mariadb-server</strong>.
[14:47:18]:<em style='color: #c66211;'>172.16.101.179</em>: Installing <strong style='color: #59a449;'>mariadb</strong> (<strong style='color: #59a449;'>galera</strong>) on <strong style='color: #59a449;'>debian</strong>.
[14:47:18]:<em style='color: #c66211;'>172.16.101.179</em>:3306: Proceeding.
[14:47:18]:<em style='color: #c66211;'>172.16.101.179</em>:3306: Waiting for dpkg lock.
[14:47:18]:<em style='color: #c66211;'>172.16.101.179</em>:3306: Installing the MySQL packages.
[14:47:14]:<em style='color: #c66211;'>172.16.101.179</em>:3306: Prepare MySQL environment (user/group).
[14:47:14]:<em style='color: #c66211;'>172.16.101.179</em>:3306: Cleaning old MySQL packages: successfully removed packages.
[14:47:06]:<em style='color: #c66211;'>172.16.101.179</em>:3306: Removing old MySQL packages from the host.
[14:46:55]:<em style='color: #c66211;'>172.16.101.179</em>: Using external repositories.
[14:46:55]:<em style='color: #c66211;'>172.16.101.179</em>:3306: Total disk space: 195.86 GiB.
[14:46:55]:<em style='color: #c66211;'>172.16.101.179</em>:3306: Free disk space: 179.64 GiB.
[14:46:55]:<em style='color: #c66211;'>172.16.101.179</em>:3306: Checking free-disk space of '<em style='color: #0b33b5;'>/var/lib/mysql</em>'.
[14:46:55]:<em style='color: #c66211;'>172.16.101.179</em>:3306: Detected memory: 7.79 GiB.
[14:46:55]:<em style='color: #c66211;'>172.16.101.179</em>:3306: Detecting total memory.
[14:46:55]:<em style='color: #c66211;'>172.16.101.179</em>:3306: <strong style='color: #59a449;'>mysql_server_id</strong> is not specified in the job, setting server_id=6001.
[14:46:55]:Loaded template '<em style='color: #007e18;'>/usr/share/cmon/templates/my.cnf.galera</em>'.
[14:46:55]:Using config template '<em style='color: #007e18;'>my.cnf.galera</em>' from '<em style='color: #0b33b5;'>/usr/share/cmon/templates</em>'.
[14:46:55]:<em style='color: #c66211;'>172.16.101.179</em>:3306: Using template '<em style='color: #007e18;'>my.cnf.galera</em>'.
[14:46:55]:Cluster type is '<strong style='color: #59a449;'>galera</strong>'.
[14:46:55]:Using <strong style='color: #59a449;'>skip_name_resolve</strong> as all servers are defined by IP.
[14:46:55]:<em style='color: #c66211;'>172.16.101.179</em>: Verifying helper packages (checking '<strong style='color: #110679;'>socat</strong>').
[14:46:55]:<em style='color: #c66211;'>172.16.101.179</em>: Finished with helper packages.
[14:46:54]:<em style='color: #c66211;'>172.16.101.179</em>: remote (172.16.101.179) retval: 100 command: DEBIAN_FRONTEND=noninteractive LC_ALL=C apt-get -o Dpkg::Options::='--force-confnew' -o Dpkg::Options::='--force-confmiss' -qq -y --force-yes install augeas-tools 2>&1 output: W: --force-yes is deprecated, use one of the options starting with --allow instead.
E: Package 'augeas-tools' has no installation candidate.
[14:46:54]:<em style='color: #c66211;'>172.16.101.179</em>: Failed to install <strong style='color: #110679;'>augeas-tools</strong>.
[14:46:54]:<em style='color: #c66211;'>172.16.101.179</em>: Installing <strong style='color: #110679;'>augeas-tools</strong>.
[14:46:54]:<em style='color: #c66211;'>172.16.101.179</em>: Installing <strong style='color: #110679;'>apt-transport-https</strong>.
[14:46:54]:<em style='color: #c66211;'>172.16.101.179</em>: Installing <strong style='color: #110679;'>openssl</strong>.
[14:46:53]:<em style='color: #c66211;'>172.16.101.179</em>: Installing <strong style='color: #110679;'>libatomic1</strong>.
[14:46:52]:<em style='color: #c66211;'>172.16.101.179</em>: Installing <strong style='color: #110679;'>tar</strong>.
[14:46:52]:<em style='color: #c66211;'>172.16.101.179</em>: Installing <strong style='color: #110679;'>iproute2</strong>.
[14:46:52]:<em style='color: #c66211;'>172.16.101.179</em>: Installing <strong style='color: #110679;'>bzip2</strong>.
[14:46:52]:<em style='color: #c66211;'>172.16.101.179</em>: remote (172.16.101.179) retval: 100 command: DEBIAN_FRONTEND=noninteractive LC_ALL=C apt-get -o Dpkg::Options::='--force-confnew' -o Dpkg::Options::='--force-confmiss' -qq -y --force-yes install libmecab2 2>&1 output: W: --force-yes is deprecated, use one of the options starting with --allow instead.
E: Package 'libmecab2' has no installation candidate.
[14:46:52]:<em style='color: #c66211;'>172.16.101.179</em>: Failed to install <strong style='color: #110679;'>libmecab2</strong>.
[14:46:52]:<em style='color: #c66211;'>172.16.101.179</em>: Installing <strong style='color: #110679;'>libmecab2</strong>.
[14:46:51]:<em style='color: #c66211;'>172.16.101.179</em>: Installing <strong style='color: #110679;'>libnuma1</strong>.
[14:46:51]:<em style='color: #c66211;'>172.16.101.179</em>: remote (172.16.101.179) retval: 100 command: DEBIAN_FRONTEND=noninteractive LC_ALL=C apt-get -o Dpkg::Options::='--force-confnew' -o Dpkg::Options::='--force-confmiss' -qq -y --force-yes install pigz 2>&1 output: W: --force-yes is deprecated, use one of the options starting with --allow instead.
E: Unable to locate package pigz.
[14:46:51]:<em style='color: #c66211;'>172.16.101.179</em>: Failed to install <strong style='color: #110679;'>pigz</strong>.
[14:46:51]:<em style='color: #c66211;'>172.16.101.179</em>: Installing <strong style='color: #110679;'>pigz</strong>.
[14:46:51]:<em style='color: #c66211;'>172.16.101.179</em>: Installing <strong style='color: #110679;'>curl</strong>.
[14:46:50]:<em style='color: #c66211;'>172.16.101.179</em>: Installing <strong style='color: #110679;'>wget</strong>.
[14:46:50]:<em style='color: #c66211;'>172.16.101.179</em>: remote (172.16.101.179) retval: 100 command: DEBIAN_FRONTEND=noninteractive LC_ALL=C apt-get -o Dpkg::Options::='--force-confnew' -o Dpkg::Options::='--force-confmiss' -qq -y --force-yes install libdbd-mysql-perl 2>&1 output: W: --force-yes is deprecated, use one of the options starting with --allow instead.
E: Package 'libdbd-mysql-perl' has no installation candidate.
[14:46:50]:<em style='color: #c66211;'>172.16.101.179</em>: Failed to install <strong style='color: #110679;'>libdbd-mysql-perl</strong>.
[14:46:50]:<em style='color: #c66211;'>172.16.101.179</em>: Installing <strong style='color: #110679;'>libdbd-mysql-perl</strong>.
[14:46:50]:<em style='color: #c66211;'>172.16.101.179</em>: Installing <strong style='color: #110679;'>libssl1.0.0</strong>.
[14:46:50]:<em style='color: #c66211;'>172.16.101.179</em>: Installing <strong style='color: #110679;'>lsb-release</strong>.
[14:46:49]:<em style='color: #c66211;'>172.16.101.179</em>: Installing <strong style='color: #110679;'>socat</strong>.
[14:46:48]:<em style='color: #c66211;'>172.16.101.179</em>: Installing <strong style='color: #110679;'>netcat-openbsd</strong>.
[14:46:48]:<em style='color: #c66211;'>172.16.101.179</em>: Installing <strong style='color: #110679;'>netcat</strong>.
[14:46:48]:<em style='color: #c66211;'>172.16.101.179</em>: Installing <strong style='color: #110679;'>libaio1</strong>.
[14:46:48]:<em style='color: #c66211;'>172.16.101.179</em>: Installing <strong style='color: #110679;'>rsync</strong>.
[14:46:47]:<em style='color: #c66211;'>172.16.101.179</em>: Installing <strong style='color: #110679;'>psmisc</strong>.
[14:46:45]:<em style='color: #c66211;'>172.16.101.179</em>: Proceeding.
[14:46:45]:<em style='color: #c66211;'>172.16.101.179</em>: Waiting for dpkg lock.
[14:46:45]:<em style='color: #c66211;'>172.16.101.179</em>: Installing helper packages.
[14:46:45]:<em style='color: #c66211;'>172.16.101.179</em>: Setting <strong style='color: #59a449;'>vm.swappiness</strong> = 1.
[14:46:45]:<em style='color: #c66211;'>172.16.101.179</em>: Tuning OS parameters.
[14:46:45]:<em style='color: #c66211;'>172.16.101.179</em>: Flushing iptables.
[14:46:43]:<em style='color: #c66211;'>172.16.101.179</em>: Disabling ufw.
[14:46:41]:<em style='color: #c66211;'>172.16.101.179</em>: Checking firewall.
[14:46:41]:<em style='color: #c66211;'>172.16.101.179</em>: Checking SELinux status (<strong style='color: #59a449;'>enabled</strong> = false).
[14:46:39]:<em style='color: #c66211;'>172.16.101.179</em>: Check and disable AppArmor.
[14:46:39]:<em style='color: #c66211;'>172.16.101.179</em>: Type is '<strong style='color: #59a449;'>debian</strong>'.
[14:46:39]:<em style='color: #c66211;'>172.16.101.179</em>: Release is '<strong style='color: #59a449;'>bionic</strong>'.
[14:46:39]:<em style='color: #c66211;'>172.16.101.179</em>: Vendor is '<strong style='color: #59a449;'>ubuntu</strong>'.
[14:46:39]:Calling job: setupServer(<em style='color: #c66211;'>172.16.101.179</em>:3306).
[14:46:39]:<strong style='color: #59a449;'>wsrep_cluster_address</strong> = 'gcomm://172.16.101.179,172.16.101.180'
[14:46:39]:Creating the cluster with the following:
[14:46:38]:<em style='color: #1abc9c;'>172.16.101.180</em>:3306: Checking if host already exists in another cluster.
[14:46:38]:<em style='color: #c66211;'>172.16.101.179</em>:3306: Checking if host already exists in another cluster.
[14:46:38]:All 2 hosts are accessible by SSH.
[14:46:38]:Using sudo password for further communication.
[14:46:38]:<em style='color: #1abc9c;'>172.16.101.180</em>: Access with ssh/sudo granted.
[14:46:37]:<em style='color: #c66211;'>172.16.101.179</em>: Access with ssh/sudo granted.
[14:46:36]:Checking ssh/sudo on 2 hosts.
[14:46:36]:Verifying job parameters.
-
Official comment
Hi,
[14:47:21]:<em style='color: #c66211;'>172.16.101.179</em>:3306: failed remote (172.16.101.179) retval: 100 command: DEBIAN_FRONTEND=noninteractive LC_ALL=C apt-get -o Dpkg::Options::='--force-confnew' -o Dpkg::Options::='--force-confmiss' -qq -y --force-yes install mariadb-server mariadb-client mariadb-common mariadb-backup percona-toolkit 2>&1 output: W: --force-yes is deprecated, use one of the options starting with --allow instead.
E: Unable to correct problems, you have held broken packages.This error means there is some sort of a package conflict which prevents ClusterControl from installing some of the required packages. What I can suggest at this time is to try and install them manually. You should have a clearer picture of what the exact issue is. Ubuntu 18.04 LTS is a supported distribution which kinda makes me think this might be a problem related to your environment. Some non-standard packages have been installed or similar. Please see if you can sort it out on your own, if not feel free to open a ticket with us and we will try to assist you with this case.
Comment actions -
Hi Ksiazek, thanks for your suggestions.
There was something wrong with the internet connection on DB servers and during the deployment of cluster some of the packages were getting skipped. So I followed your suggestion and installed them manually via apt-get and retried. Now everything is working fine.
Thanks again for the help, here is a screenshot of my successfully deployed and imported clusters...
Please sign in to leave a comment.
Comments
2 comments