Unable to add galera (MySQL) cluster and 127.0.0.1:9500 connection refused error

June 22, 2019 19:45

I am facing an issue in clusterControl and I have searched not only the community support but also online for many hours and can't figure it out, I will be grateful if anyone can guide me in right direction.

I did following steps

1: installed cluster control on ubuntu 18.4 via ./install-cc (installation went smooth)

2: created two other ubuntu 18.4 machines for db cluster

3: created ssh users on all 3 servers with same username and password and activated passwordless ssh on clusterControl machine (also did that ssh-copy-id [IP_ADDRESS] )

4: access http:[ip of clusterControl]/clustercontrol/, created user and logged in

5: right after login i got this error alert message on top right corner "failed to connection 127.0.0.1 port 9500, connection refused"

6: clicked on deploy and selected galera, on first step entered ssh username+password and cluster name, clicked next, and got the error message "Something went wrong"

7: closed the error message and added db node IP address in "Add Node" section but it is not connecting to the node, node keeps spinning that preloader and doesn't complete the node adding process.

I have already watched video tuts by Calen Curry on youtube and followed instruction manuals form website but still can't figure it out.

(badly need help with points # 5, 6 and 7)

Comments

3 comments

Official comment
Krzysztof Ksiazek

June 23, 2019 06:32
Hi,

The error you got makes it look like there is some sort of a connectivity issue going on. Please ensure that the cmon service is up and running on the host and that you can connect to localhost (127.0.0.1) on port 9500. If needed, open this port on the firewall. You can test it by trying to access 127.0.0.1:9500 using for example telnet.

If this won't help, you can always open a ticket with us (support.severalnines.com) and we will try to assist you with this. We may require more data to diagnose this case, data which should not be shared publicly.

Thanks,

Krzysztof
Comment actions Permalink
Waqar Akbar

June 23, 2019 10:02

Edited
Thank Krzysztof,

You were right, 127.0.0.1:9500 was not accessible because auto installation of clusterControl (./install-cc) on my server (ubuntu 18.4) was not installing clustercontrol-controller correctly so I have to opt for offline installation method. It worked and now clusterControl runs without errors.

Now while I am adding a new cluster (MySQL Galera), the job always fails. I have entered required information in step:1 and step:2 as shown in these screenshots (followed Caleb curry's youtube videos and clusterControl docs). The job output is also at the end of this comment. Passwordless SSH is setup with same username/password on all three servers (1 clusterControl and 2 database servers).

I have restarted the job a few times but it always fails.

I have tried it with both, MariaDB and PerConaXtraDB just to sure, both give same result.

Step: 1

Step: 2

Job Output:

[14:47:21]:172.16.101.179:3306: Failed to setup MySql server.
[14:47:21]:172.16.101.179: Setup server failure, see the previous messages.
[14:47:21]:172.16.101.179:3306: failed remote (172.16.101.179) retval: 100 command: DEBIAN_FRONTEND=noninteractive LC_ALL=C apt-get -o Dpkg::Options::='--force-confnew' -o Dpkg::Options::='--force-confmiss' -qq -y --force-yes install mariadb-server mariadb-client mariadb-common mariadb-backup percona-toolkit 2>&1 output: W: --force-yes is deprecated, use one of the options starting with --allow instead.
E: Unable to correct problems, you have held broken packages.
[14:47:21]: output: W: --force-yes is deprecated, use one of the options starting with --allow instead.
E: Unable to correct problems, you have held broken packages.
[14:47:21]: command: DEBIAN_FRONTEND=noninteractive LC_ALL=C apt-get -o Dpkg::Options::='--force-confnew' -o Dpkg::Options::='--force-confmiss' -qq -y --force-yes install mariadb-server mariadb-client mariadb-common mariadb-backup percona-toolkit 2>&1
[14:47:21]:Failed to install package.
[14:47:20]:172.16.101.179: Installing mariadb-server mariadb-client mariadb-common mariadb-backup percona-toolkit .
[14:47:20]: output: W: --force-yes is deprecated, use one of the options starting with --allow instead.
E: Unable to correct problems, you have held broken packages.
[14:47:20]: command: DEBIAN_FRONTEND=noninteractive LC_ALL=C apt-get -o Dpkg::Options::='--force-confnew' -o Dpkg::Options::='--force-confmiss' -qq -y --force-yes install mariadb-server mariadb-client mariadb-common mariadb-backup percona-toolkit 2>&1
[14:47:20]:Failed to install package.
[14:47:20]:172.16.101.179: Installing mariadb-server mariadb-client mariadb-common mariadb-backup percona-toolkit .
[14:47:20]:172.16.101.179: Installing percona-toolkit.
[14:47:20]:172.16.101.179: Installing mariadb-backup.
[14:47:20]:172.16.101.179: Installing mariadb-common.
[14:47:20]:172.16.101.179: Installing mariadb-client.
[14:47:20]:172.16.101.179: Installing mariadb-server.
[14:47:18]:172.16.101.179: Installing mariadb (galera) on debian.
[14:47:18]:172.16.101.179:3306: Proceeding.
[14:47:18]:172.16.101.179:3306: Waiting for dpkg lock.
[14:47:18]:172.16.101.179:3306: Installing the MySQL packages.
[14:47:14]:172.16.101.179:3306: Prepare MySQL environment (user/group).
[14:47:14]:172.16.101.179:3306: Cleaning old MySQL packages: successfully removed packages.
[14:47:06]:172.16.101.179:3306: Removing old MySQL packages from the host.
[14:46:55]:172.16.101.179: Using external repositories.
[14:46:55]:172.16.101.179:3306: Total disk space: 195.86 GiB.
[14:46:55]:172.16.101.179:3306: Free disk space: 179.64 GiB.
[14:46:55]:172.16.101.179:3306: Checking free-disk space of '/var/lib/mysql'.
[14:46:55]:172.16.101.179:3306: Detected memory: 7.79 GiB.
[14:46:55]:172.16.101.179:3306: Detecting total memory.
[14:46:55]:172.16.101.179:3306: mysql_server_id is not specified in the job, setting server_id=6001.
[14:46:55]:Loaded template '/usr/share/cmon/templates/my.cnf.galera'.
[14:46:55]:Using config template 'my.cnf.galera' from '/usr/share/cmon/templates'.
[14:46:55]:172.16.101.179:3306: Using template 'my.cnf.galera'.
[14:46:55]:Cluster type is 'galera'.
[14:46:55]:Using skip_name_resolve as all servers are defined by IP.
[14:46:55]:172.16.101.179: Verifying helper packages (checking 'socat').
[14:46:55]:172.16.101.179: Finished with helper packages.
[14:46:54]:172.16.101.179: remote (172.16.101.179) retval: 100 command: DEBIAN_FRONTEND=noninteractive LC_ALL=C apt-get -o Dpkg::Options::='--force-confnew' -o Dpkg::Options::='--force-confmiss' -qq -y --force-yes install augeas-tools 2>&1 output: W: --force-yes is deprecated, use one of the options starting with --allow instead.
E: Package 'augeas-tools' has no installation candidate.
[14:46:54]:172.16.101.179: Failed to install augeas-tools.
[14:46:54]:172.16.101.179: Installing augeas-tools.
[14:46:54]:172.16.101.179: Installing apt-transport-https.
[14:46:54]:172.16.101.179: Installing openssl.
[14:46:53]:172.16.101.179: Installing libatomic1.
[14:46:52]:172.16.101.179: Installing tar.
[14:46:52]:172.16.101.179: Installing iproute2.
[14:46:52]:172.16.101.179: Installing bzip2.
[14:46:52]:172.16.101.179: remote (172.16.101.179) retval: 100 command: DEBIAN_FRONTEND=noninteractive LC_ALL=C apt-get -o Dpkg::Options::='--force-confnew' -o Dpkg::Options::='--force-confmiss' -qq -y --force-yes install libmecab2 2>&1 output: W: --force-yes is deprecated, use one of the options starting with --allow instead.
E: Package 'libmecab2' has no installation candidate.
[14:46:52]:172.16.101.179: Failed to install libmecab2.
[14:46:52]:172.16.101.179: Installing libmecab2.
[14:46:51]:172.16.101.179: Installing libnuma1.
[14:46:51]:172.16.101.179: remote (172.16.101.179) retval: 100 command: DEBIAN_FRONTEND=noninteractive LC_ALL=C apt-get -o Dpkg::Options::='--force-confnew' -o Dpkg::Options::='--force-confmiss' -qq -y --force-yes install pigz 2>&1 output: W: --force-yes is deprecated, use one of the options starting with --allow instead.
E: Unable to locate package pigz.
[14:46:51]:172.16.101.179: Failed to install pigz.
[14:46:51]:172.16.101.179: Installing pigz.
[14:46:51]:172.16.101.179: Installing curl.
[14:46:50]:172.16.101.179: Installing wget.
[14:46:50]:172.16.101.179: remote (172.16.101.179) retval: 100 command: DEBIAN_FRONTEND=noninteractive LC_ALL=C apt-get -o Dpkg::Options::='--force-confnew' -o Dpkg::Options::='--force-confmiss' -qq -y --force-yes install libdbd-mysql-perl 2>&1 output: W: --force-yes is deprecated, use one of the options starting with --allow instead.
E: Package 'libdbd-mysql-perl' has no installation candidate.
[14:46:50]:172.16.101.179: Failed to install libdbd-mysql-perl.
[14:46:50]:172.16.101.179: Installing libdbd-mysql-perl.
[14:46:50]:172.16.101.179: Installing libssl1.0.0.
[14:46:50]:172.16.101.179: Installing lsb-release.
[14:46:49]:172.16.101.179: Installing socat.
[14:46:48]:172.16.101.179: Installing netcat-openbsd.
[14:46:48]:172.16.101.179: Installing netcat.
[14:46:48]:172.16.101.179: Installing libaio1.
[14:46:48]:172.16.101.179: Installing rsync.
[14:46:47]:172.16.101.179: Installing psmisc.
[14:46:45]:172.16.101.179: Proceeding.
[14:46:45]:172.16.101.179: Waiting for dpkg lock.
[14:46:45]:172.16.101.179: Installing helper packages.
[14:46:45]:172.16.101.179: Setting vm.swappiness = 1.
[14:46:45]:172.16.101.179: Tuning OS parameters.
[14:46:45]:172.16.101.179: Flushing iptables.
[14:46:43]:172.16.101.179: Disabling ufw.
[14:46:41]:172.16.101.179: Checking firewall.
[14:46:41]:172.16.101.179: Checking SELinux status (enabled = false).
[14:46:39]:172.16.101.179: Check and disable AppArmor.
[14:46:39]:172.16.101.179: Type is 'debian'.
[14:46:39]:172.16.101.179: Release is 'bionic'.
[14:46:39]:172.16.101.179: Vendor is 'ubuntu'.
[14:46:39]:Calling job: setupServer(172.16.101.179:3306).
[14:46:39]:wsrep_cluster_address = 'gcomm://172.16.101.179,172.16.101.180'
[14:46:39]:Creating the cluster with the following:
[14:46:38]:172.16.101.180:3306: Checking if host already exists in another cluster.
[14:46:38]:172.16.101.179:3306: Checking if host already exists in another cluster.
[14:46:38]:All 2 hosts are accessible by SSH.
[14:46:38]:Using sudo password for further communication.
[14:46:38]:172.16.101.180: Access with ssh/sudo granted.
[14:46:37]:172.16.101.179: Access with ssh/sudo granted.
[14:46:36]:Checking ssh/sudo on 2 hosts.
[14:46:36]:Verifying job parameters.
0

Comment actions Permalink
Krzysztof Ksiazek

June 23, 2019 16:23
Hi,
```
[14:47:21]:172.16.101.179:3306: failed remote (172.16.101.179) retval: 100 command: DEBIAN_FRONTEND=noninteractive LC_ALL=C apt-get -o Dpkg::Options::='--force-confnew' -o Dpkg::Options::='--force-confmiss' -qq -y --force-yes install mariadb-server mariadb-client mariadb-common mariadb-backup percona-toolkit 2>&1 output: W: --force-yes is deprecated, use one of the options starting with --allow instead.
E: Unable to correct problems, you have held broken packages.
```
This error means there is some sort of a package conflict which prevents ClusterControl from installing some of the required packages. What I can suggest at this time is to try and install them manually. You should have a clearer picture of what the exact issue is. Ubuntu 18.04 LTS is a supported distribution which kinda makes me think this might be a problem related to your environment. Some non-standard packages have been installed or similar. Please see if you can sort it out on your own, if not feel free to open a ticket with us and we will try to assist you with this case.
0

Comment actions Permalink

Please sign in to leave a comment.

Comments

Didn't find what you were looking for?