Not able to setup new cluster

Hi all,

I try to get the 3 mariadb galera test machines setup in ClusterControl but if fails every time.
All settings are correct.
But every time I get the error message that the cluster setup fails.

Any idea what can be wrong here?

Cheers
Christian

This is the job worklog:

[16:46:31]:New cluster setup failure: cluster 49 is not running.
[16:46:31]:127.0.0.1:/var/log/cmon_49.log:
----8<-----8<-----8<-----
2019-04-04 14:46:25 : (INFO) Creating host 192.168.10.168(192.168.10.168).
2019-04-04 14:46:26 : (INFO) Creating host 192.168.10.171(192.168.10.171).
2019-04-04 14:46:26 : (INFO) Creating host 192.168.10.172(192.168.10.172).
2019-04-04 14:46:26 : (INFO) Configuration loaded.
2019-04-04 14:46:26 : (INFO) Looking up (1): galeracluster01
2019-04-04 14:46:26 : (INFO) Testing connection to mysqld...
2019-04-04 14:46:26 : (WARNING) Alarm raised: License expiration.
2019-04-04 14:46:26 : (INFO) Enterprise version
2019-04-04 14:46:26 : (INFO) Found 'cmon' schema version 107011.
2019-04-04 14:46:26 : (INFO) Running cmon schema hot-fixes.
2019-04-04 14:46:27 : (INFO) Schema auto-upgrade succeed (version 107011).
2019-04-04 14:46:27 : (INFO) Checked tables - seems ok
2019-04-04 14:46:27 : (INFO) Schema looks complete.
2019-04-04 14:46:30 : (INFO) Managed cluster has been registered - registered cluster id=49
2019-04-04 14:46:30 : (INFO) Starting cmon version 1.7.1.3085 with the following parameters:
mysql-hostname=127.0.0.1
mysql-port=3306
cluster-id=49
cluster-type=galera
hostname=galeracluster01
db_hourly_stats_collection_interval=5
db_stats_collection_interval=30
host_stats_collection_interval=30
ssh_port=22
ssh_identity=/home/sysadmin/.ssh/id_rsa
os_user=sysadmin
os_user_home=/home/sysadmin
os=debian
enable_cluster_autorecovery=0
enable_node_autorecovery=0
2019-04-04 14:46:30 : (INFO) If that doesn't look correct, kill cmon and restart with -? for help on the parameters, or change the params in /etc/init.d/cmon
2019-04-04 14:46:30 : (INFO) Installed package versions:
clustercontrol-1.7.1-5812
clustercontrol-cmonapi-1.7.1-338
clustercontrol-controller-1.7.1-3085

2019-04-04 14:46:30 : (INFO) Creating staging area on controller.
2019-04-04 14:46:30 : (INFO) Creating host galeracluster01(192.168.10.168).
2019-04-04 14:46:30 : (ERROR) Failed to create host ('galeracluster01', ip: '192.168.10.168')

----8<-----8<-----8<-----

[16:46:26]:Waiting until the initial cluster starts up.
[16:46:26]:Sending SIGHUP to the controller process.
[16:46:26]:Configuration written to '/etc/cmon.d/cmon_49.cnf'.
[16:46:25]:Generating and writing the new configuration.
[16:46:25]:192.168.10.168:3306: Detected OS = 'debian'
[16:46:25]:192.168.10.168: Detecting the OS.
[16:46:25]:192.168.10.172:3306: The datadir is /var/lib/mysql/.
[16:46:25]:Connected succesfully: 'cmon@galeracluster01' -> '192.168.10.172:3306'.
[16:46:25]:192.168.10.171:3306: The datadir is /var/lib/mysql/.
[16:46:25]:Connected succesfully: 'cmon@galeracluster01' -> '192.168.10.171:3306'.
[16:46:25]:192.168.10.168:3306: The datadir is /var/lib/mysql/.
[16:46:25]:Connected succesfully: 'cmon@galeracluster01' -> '192.168.10.168:3306'.
[16:46:25]:Determining data directories.
[16:46:23]:192.168.10.172: Granting cmon.
[16:46:21]:192.168.10.171: Granting cmon.
[16:46:20]:192.168.10.168: Granting cmon.
[16:46:20]:192.168.10.172:3306: Node is synced.
[16:46:19]:192.168.10.171:3306: Node is synced.
[16:46:19]:192.168.10.168:3306: Node is synced.
[16:46:18]:Granting controller (galeracluster01).
[16:46:18]:Controller 'galeracluster01' could not be resolved. Probably already a valid FQDN.
[16:46:18]:Checking if resolve of 'galeracluster01' is needed, and attempting to resolve to a hostname
[16:46:17]:Detected that skip_name_resolve is not used on the target server(s).
[16:46:16]:Check SELinux statuses.
[16:46:16]:All 3 hosts are accessible by SSH.
[16:46:16]:Using sudo password for further communication.
[16:46:16]:192.168.10.172: Access with ssh/sudo granted.
[16:46:14]:192.168.10.171: Access with ssh/sudo granted.
[16:46:14]:192.168.10.168: already checked 5 seconds ago.
[16:46:13]:Checking ssh/sudo on 3 hosts.
[16:46:13]:Checking that nodes are not in another cluster.
[16:46:13]:Found in total 3 nodes.
[16:46:13]:Found node: '192.168.10.172'
[16:46:13]:Found node: '192.168.10.171'
[16:46:13]:Found node: '192.168.10.168'
[16:46:13]:192.168.10.172: wsrep_node_address=192.168.10.172
[16:46:11]:192.168.10.172: Checking wsrep_node_address.
[16:46:11]:192.168.10.171: wsrep_node_address=192.168.10.171
[16:46:10]:192.168.10.171: Checking wsrep_node_address.
[16:46:10]:192.168.10.168: wsrep_node_address=192.168.10.168
[16:46:10]:192.168.10.168: Checking wsrep_node_address.
[16:46:09]:192.168.10.168:3306: Getting node list from the MySQL server.
[16:46:09]:192.168.10.168:3306: Verifying the MySQL user/password.
[16:46:09]:Using sudo password for further communication.
[16:46:09]:192.168.10.168: Access with ssh/sudo granted.
[16:46:09]:192.168.10.168: Checking ssh/sudo.
[16:46:09]:Adding existing MySQL cluster.
[16:46:09]:Cmon user with ID 1 was not found: Error finding job owner: User id 1 is not found.
[16:46:09]:Generating configuration for cluster test_cluster.
[16:46:09]:Verifying controller host and cmon password.
[16:46:09]:Adding cluster with type 'galera', vendor 'mariadb'.
Job spec: {
"command": "add_cluster",
"job_data":
{
"api_id": 1,
"cluster_name": "test_cluster",
"cluster_type": "galera",
"config_template": "my.cnf.galera",
"datadir": "/var/lib/mysql",
"db_password": "123456789",
"db_user": "root",
"disable_firewall": true,
"disable_selinux": true,
"enable_cluster_autorecovery": false,
"enable_information_schema_queries": false,
"enable_node_autorecovery": false,
"enable_nodes_autodetect": true,
"galera_version": "2.x",
"generate_token": true,
"install_software": true,
"monitored_mysql_port": "3306",
"nodes": [
{
"hostname": "192.168.10.168",
"port": "3306"
} ],
"port": "3306",
"ssh_keyfile": "/home/sysadmin/.ssh/id_rsa",
"ssh_port": "22",
"ssh_user": "sysadmin",
"sudo_password": "I=Boss!1200",
"tag": "",
"user_id": 1,
"vendor": "mariadb",
"version": "10.1"
}
}