Unable to add galera (MySQL) cluster and 127.0.0.1:9500 connection refused error

Comments

3 comments

  • Official comment
    Avatar
    Krzysztof Ksiazek

    Hi,

    The error you got makes it look like there is some sort of a connectivity issue going on. Please ensure that the cmon service is up and running on the host and that you can connect to localhost (127.0.0.1) on port 9500. If needed, open this port on the firewall. You can test it by trying to access 127.0.0.1:9500 using for example telnet.

    If this won't help, you can always open a ticket with us (support.severalnines.com) and we will try to assist you with this. We may require more data to diagnose this case, data which should not be shared publicly.

    Thanks,

    Krzysztof

  • Avatar
    Waqar Akbar

    Thank Krzysztof, 

    You were right, 127.0.0.1:9500  was not accessible because auto installation of clusterControl (./install-cc) on my server (ubuntu 18.4) was not installing clustercontrol-controller correctly so I have to opt for offline installation method. It worked and now clusterControl runs without errors.

    Now while I am adding a new cluster (MySQL Galera), the job always fails. I have entered required information in step:1 and step:2 as shown in these screenshots (followed Caleb curry's youtube videos and clusterControl docs). The job output is also at the end of this comment. Passwordless SSH is setup with same username/password on all three servers (1 clusterControl and 2 database servers).

    I have restarted the job a few times but it always fails.

    I have tried it with both, MariaDB and PerConaXtraDB just to sure, both give same result.

     

    Step: 1

     

     

    Step: 2

     

     

    Job Output:

    [14:47:21]:<em style='color: #c66211;'>172.16.101.179</em>:3306: Failed to setup MySql server.
    [14:47:21]:<em style='color: #c66211;'>172.16.101.179</em>: Setup server failure, see the previous messages.
    [14:47:21]:<em style='color: #c66211;'>172.16.101.179</em>:3306: failed remote (172.16.101.179) retval: 100 command: DEBIAN_FRONTEND=noninteractive LC_ALL=C apt-get -o Dpkg::Options::='--force-confnew' -o Dpkg::Options::='--force-confmiss' -qq -y --force-yes install mariadb-server mariadb-client mariadb-common mariadb-backup percona-toolkit 2>&1 output: W: --force-yes is deprecated, use one of the options starting with --allow instead.
    E: Unable to correct problems, you have held broken packages.
    [14:47:21]: output: W: --force-yes is deprecated, use one of the options starting with --allow instead.
    E: Unable to correct problems, you have held broken packages.
    [14:47:21]: command: DEBIAN_FRONTEND=noninteractive LC_ALL=C apt-get -o Dpkg::Options::='--force-confnew' -o Dpkg::Options::='--force-confmiss' -qq -y --force-yes install mariadb-server mariadb-client mariadb-common mariadb-backup percona-toolkit 2>&1
    [14:47:21]:Failed to install package.
    [14:47:20]:<em style='color: #c66211;'>172.16.101.179</em>: Installing <strong style='color: #110679;'>mariadb-server mariadb-client mariadb-common mariadb-backup percona-toolkit </strong>.
    [14:47:20]: output: W: --force-yes is deprecated, use one of the options starting with --allow instead.
    E: Unable to correct problems, you have held broken packages.
    [14:47:20]: command: DEBIAN_FRONTEND=noninteractive LC_ALL=C apt-get -o Dpkg::Options::='--force-confnew' -o Dpkg::Options::='--force-confmiss' -qq -y --force-yes install mariadb-server mariadb-client mariadb-common mariadb-backup percona-toolkit 2>&1
    [14:47:20]:Failed to install package.
    [14:47:20]:<em style='color: #c66211;'>172.16.101.179</em>: Installing <strong style='color: #110679;'>mariadb-server mariadb-client mariadb-common mariadb-backup percona-toolkit </strong>.
    [14:47:20]:<em style='color: #c66211;'>172.16.101.179</em>: Installing <strong style='color: #110679;'>percona-toolkit</strong>.
    [14:47:20]:<em style='color: #c66211;'>172.16.101.179</em>: Installing <strong style='color: #110679;'>mariadb-backup</strong>.
    [14:47:20]:<em style='color: #c66211;'>172.16.101.179</em>: Installing <strong style='color: #110679;'>mariadb-common</strong>.
    [14:47:20]:<em style='color: #c66211;'>172.16.101.179</em>: Installing <strong style='color: #110679;'>mariadb-client</strong>.
    [14:47:20]:<em style='color: #c66211;'>172.16.101.179</em>: Installing <strong style='color: #110679;'>mariadb-server</strong>.
    [14:47:18]:<em style='color: #c66211;'>172.16.101.179</em>: Installing <strong style='color: #59a449;'>mariadb</strong> (<strong style='color: #59a449;'>galera</strong>) on <strong style='color: #59a449;'>debian</strong>.
    [14:47:18]:<em style='color: #c66211;'>172.16.101.179</em>:3306: Proceeding.
    [14:47:18]:<em style='color: #c66211;'>172.16.101.179</em>:3306: Waiting for dpkg lock.
    [14:47:18]:<em style='color: #c66211;'>172.16.101.179</em>:3306: Installing the MySQL packages.
    [14:47:14]:<em style='color: #c66211;'>172.16.101.179</em>:3306: Prepare MySQL environment (user/group).
    [14:47:14]:<em style='color: #c66211;'>172.16.101.179</em>:3306: Cleaning old MySQL packages: successfully removed packages.
    [14:47:06]:<em style='color: #c66211;'>172.16.101.179</em>:3306: Removing old MySQL packages from the host.
    [14:46:55]:<em style='color: #c66211;'>172.16.101.179</em>: Using external repositories.
    [14:46:55]:<em style='color: #c66211;'>172.16.101.179</em>:3306: Total disk space: 195.86 GiB.
    [14:46:55]:<em style='color: #c66211;'>172.16.101.179</em>:3306: Free disk space: 179.64 GiB.
    [14:46:55]:<em style='color: #c66211;'>172.16.101.179</em>:3306: Checking free-disk space of '<em style='color: #0b33b5;'>/var/lib/mysql</em>'.
    [14:46:55]:<em style='color: #c66211;'>172.16.101.179</em>:3306: Detected memory: 7.79 GiB.
    [14:46:55]:<em style='color: #c66211;'>172.16.101.179</em>:3306: Detecting total memory.
    [14:46:55]:<em style='color: #c66211;'>172.16.101.179</em>:3306: <strong style='color: #59a449;'>mysql_server_id</strong> is not specified in the job, setting server_id=6001.
    [14:46:55]:Loaded template '<em style='color: #007e18;'>/usr/share/cmon/templates/my.cnf.galera</em>'.
    [14:46:55]:Using config template '<em style='color: #007e18;'>my.cnf.galera</em>' from '<em style='color: #0b33b5;'>/usr/share/cmon/templates</em>'.
    [14:46:55]:<em style='color: #c66211;'>172.16.101.179</em>:3306: Using template '<em style='color: #007e18;'>my.cnf.galera</em>'.
    [14:46:55]:Cluster type is '<strong style='color: #59a449;'>galera</strong>'.
    [14:46:55]:Using <strong style='color: #59a449;'>skip_name_resolve</strong> as all servers are defined by IP.
    [14:46:55]:<em style='color: #c66211;'>172.16.101.179</em>: Verifying helper packages (checking '<strong style='color: #110679;'>socat</strong>').
    [14:46:55]:<em style='color: #c66211;'>172.16.101.179</em>: Finished with helper packages.
    [14:46:54]:<em style='color: #c66211;'>172.16.101.179</em>: remote (172.16.101.179) retval: 100 command: DEBIAN_FRONTEND=noninteractive LC_ALL=C apt-get -o Dpkg::Options::='--force-confnew' -o Dpkg::Options::='--force-confmiss' -qq -y --force-yes install augeas-tools 2>&1 output: W: --force-yes is deprecated, use one of the options starting with --allow instead.
    E: Package 'augeas-tools' has no installation candidate.
    [14:46:54]:<em style='color: #c66211;'>172.16.101.179</em>: Failed to install <strong style='color: #110679;'>augeas-tools</strong>.
    [14:46:54]:<em style='color: #c66211;'>172.16.101.179</em>: Installing <strong style='color: #110679;'>augeas-tools</strong>.
    [14:46:54]:<em style='color: #c66211;'>172.16.101.179</em>: Installing <strong style='color: #110679;'>apt-transport-https</strong>.
    [14:46:54]:<em style='color: #c66211;'>172.16.101.179</em>: Installing <strong style='color: #110679;'>openssl</strong>.
    [14:46:53]:<em style='color: #c66211;'>172.16.101.179</em>: Installing <strong style='color: #110679;'>libatomic1</strong>.
    [14:46:52]:<em style='color: #c66211;'>172.16.101.179</em>: Installing <strong style='color: #110679;'>tar</strong>.
    [14:46:52]:<em style='color: #c66211;'>172.16.101.179</em>: Installing <strong style='color: #110679;'>iproute2</strong>.
    [14:46:52]:<em style='color: #c66211;'>172.16.101.179</em>: Installing <strong style='color: #110679;'>bzip2</strong>.
    [14:46:52]:<em style='color: #c66211;'>172.16.101.179</em>: remote (172.16.101.179) retval: 100 command: DEBIAN_FRONTEND=noninteractive LC_ALL=C apt-get -o Dpkg::Options::='--force-confnew' -o Dpkg::Options::='--force-confmiss' -qq -y --force-yes install libmecab2 2>&1 output: W: --force-yes is deprecated, use one of the options starting with --allow instead.
    E: Package 'libmecab2' has no installation candidate.
    [14:46:52]:<em style='color: #c66211;'>172.16.101.179</em>: Failed to install <strong style='color: #110679;'>libmecab2</strong>.
    [14:46:52]:<em style='color: #c66211;'>172.16.101.179</em>: Installing <strong style='color: #110679;'>libmecab2</strong>.
    [14:46:51]:<em style='color: #c66211;'>172.16.101.179</em>: Installing <strong style='color: #110679;'>libnuma1</strong>.
    [14:46:51]:<em style='color: #c66211;'>172.16.101.179</em>: remote (172.16.101.179) retval: 100 command: DEBIAN_FRONTEND=noninteractive LC_ALL=C apt-get -o Dpkg::Options::='--force-confnew' -o Dpkg::Options::='--force-confmiss' -qq -y --force-yes install pigz 2>&1 output: W: --force-yes is deprecated, use one of the options starting with --allow instead.
    E: Unable to locate package pigz.
    [14:46:51]:<em style='color: #c66211;'>172.16.101.179</em>: Failed to install <strong style='color: #110679;'>pigz</strong>.
    [14:46:51]:<em style='color: #c66211;'>172.16.101.179</em>: Installing <strong style='color: #110679;'>pigz</strong>.
    [14:46:51]:<em style='color: #c66211;'>172.16.101.179</em>: Installing <strong style='color: #110679;'>curl</strong>.
    [14:46:50]:<em style='color: #c66211;'>172.16.101.179</em>: Installing <strong style='color: #110679;'>wget</strong>.
    [14:46:50]:<em style='color: #c66211;'>172.16.101.179</em>: remote (172.16.101.179) retval: 100 command: DEBIAN_FRONTEND=noninteractive LC_ALL=C apt-get -o Dpkg::Options::='--force-confnew' -o Dpkg::Options::='--force-confmiss' -qq -y --force-yes install libdbd-mysql-perl 2>&1 output: W: --force-yes is deprecated, use one of the options starting with --allow instead.
    E: Package 'libdbd-mysql-perl' has no installation candidate.
    [14:46:50]:<em style='color: #c66211;'>172.16.101.179</em>: Failed to install <strong style='color: #110679;'>libdbd-mysql-perl</strong>.
    [14:46:50]:<em style='color: #c66211;'>172.16.101.179</em>: Installing <strong style='color: #110679;'>libdbd-mysql-perl</strong>.
    [14:46:50]:<em style='color: #c66211;'>172.16.101.179</em>: Installing <strong style='color: #110679;'>libssl1.0.0</strong>.
    [14:46:50]:<em style='color: #c66211;'>172.16.101.179</em>: Installing <strong style='color: #110679;'>lsb-release</strong>.
    [14:46:49]:<em style='color: #c66211;'>172.16.101.179</em>: Installing <strong style='color: #110679;'>socat</strong>.
    [14:46:48]:<em style='color: #c66211;'>172.16.101.179</em>: Installing <strong style='color: #110679;'>netcat-openbsd</strong>.
    [14:46:48]:<em style='color: #c66211;'>172.16.101.179</em>: Installing <strong style='color: #110679;'>netcat</strong>.
    [14:46:48]:<em style='color: #c66211;'>172.16.101.179</em>: Installing <strong style='color: #110679;'>libaio1</strong>.
    [14:46:48]:<em style='color: #c66211;'>172.16.101.179</em>: Installing <strong style='color: #110679;'>rsync</strong>.
    [14:46:47]:<em style='color: #c66211;'>172.16.101.179</em>: Installing <strong style='color: #110679;'>psmisc</strong>.
    [14:46:45]:<em style='color: #c66211;'>172.16.101.179</em>: Proceeding.
    [14:46:45]:<em style='color: #c66211;'>172.16.101.179</em>: Waiting for dpkg lock.
    [14:46:45]:<em style='color: #c66211;'>172.16.101.179</em>: Installing helper packages.
    [14:46:45]:<em style='color: #c66211;'>172.16.101.179</em>: Setting <strong style='color: #59a449;'>vm.swappiness</strong> = 1.
    [14:46:45]:<em style='color: #c66211;'>172.16.101.179</em>: Tuning OS parameters.
    [14:46:45]:<em style='color: #c66211;'>172.16.101.179</em>: Flushing iptables.
    [14:46:43]:<em style='color: #c66211;'>172.16.101.179</em>: Disabling ufw.
    [14:46:41]:<em style='color: #c66211;'>172.16.101.179</em>: Checking firewall.
    [14:46:41]:<em style='color: #c66211;'>172.16.101.179</em>: Checking SELinux status (<strong style='color: #59a449;'>enabled</strong> = false).
    [14:46:39]:<em style='color: #c66211;'>172.16.101.179</em>: Check and disable AppArmor.
    [14:46:39]:<em style='color: #c66211;'>172.16.101.179</em>: Type is '<strong style='color: #59a449;'>debian</strong>'.
    [14:46:39]:<em style='color: #c66211;'>172.16.101.179</em>: Release is '<strong style='color: #59a449;'>bionic</strong>'.
    [14:46:39]:<em style='color: #c66211;'>172.16.101.179</em>: Vendor is '<strong style='color: #59a449;'>ubuntu</strong>'.
    [14:46:39]:Calling job: setupServer(<em style='color: #c66211;'>172.16.101.179</em>:3306).
    [14:46:39]:<strong style='color: #59a449;'>wsrep_cluster_address</strong> = 'gcomm://172.16.101.179,172.16.101.180'
    [14:46:39]:Creating the cluster with the following:
    [14:46:38]:<em style='color: #1abc9c;'>172.16.101.180</em>:3306: Checking if host already exists in another cluster.
    [14:46:38]:<em style='color: #c66211;'>172.16.101.179</em>:3306: Checking if host already exists in another cluster.
    [14:46:38]:All 2 hosts are accessible by SSH.
    [14:46:38]:Using sudo password for further communication.
    [14:46:38]:<em style='color: #1abc9c;'>172.16.101.180</em>: Access with ssh/sudo granted.
    [14:46:37]:<em style='color: #c66211;'>172.16.101.179</em>: Access with ssh/sudo granted.
    [14:46:36]:Checking ssh/sudo on 2 hosts.
    [14:46:36]:Verifying job parameters.

  • Avatar
    Krzysztof Ksiazek

    Hi,

    [14:47:21]:<em style='color: #c66211;'>172.16.101.179</em>:3306: failed remote (172.16.101.179) retval: 100 command: DEBIAN_FRONTEND=noninteractive LC_ALL=C apt-get -o Dpkg::Options::='--force-confnew' -o Dpkg::Options::='--force-confmiss' -qq -y --force-yes install mariadb-server mariadb-client mariadb-common mariadb-backup percona-toolkit 2>&1 output: W: --force-yes is deprecated, use one of the options starting with --allow instead.
    E: Unable to correct problems, you have held broken packages.

    This error means there is some sort of a package conflict which prevents ClusterControl from installing some of the required packages. What I can suggest at this time is to try and install them manually. You should have a clearer picture of what the exact issue is. Ubuntu 18.04 LTS is a supported distribution which kinda makes me think this might be a problem related to your environment. Some non-standard packages have been installed or similar. Please see if you can sort it out on your own, if not feel free to open a ticket with us and we will try to assist you with this case.

Please sign in to leave a comment.

Powered by Zendesk