Add existing MongoDB Sharded Cluster fails
Add existing MongoDB Sharded Cluster fails
Here is the log.
Please anyone give me some advice.~
It stops on this line..
==> failed to add host to cluster. Couldn't allocate hostId for host
[17:22:24]:Adding existing cluster has failed.
[17:22:24]:<em style='color: #2ecc71;'>ms1</em>:27017: failed to add host to cluster. Couldn't allocate hostId for host (ms1[172.16.2.29], cid=10)
[17:22:24]:<em style='color: #2ecc71;'>ms1</em>:27017: registering this node with port 27017
[17:22:24]:<em style='color: #877d0f;'>cf2</em>:4000: host added to cluster.
[17:22:24]:<em style='color: #877d0f;'>cf2</em>:4000: registering this node with port 4000
[17:22:24]:<em style='color: #da0bf3;'>cf1</em>:4000: host added to cluster.
[17:22:24]:<em style='color: #da0bf3;'>cf1</em>:4000: registering this node with port 4000
[17:22:24]:<em style='color: #c0392b;'>h3</em>:27020: host added to cluster.
[17:22:24]:<em style='color: #c0392b;'>h3</em>:27020: registering this node with port 27020
[17:22:24]:<em style='color: #1abc9c;'>h2</em>:27020: host added to cluster.
[17:22:24]:<em style='color: #1abc9c;'>h2</em>:27020: registering this node with port 27020
[17:22:24]:<em style='color: #2ecc71;'>h1</em>:27020: host added to cluster.
[17:22:24]:<em style='color: #2ecc71;'>h1</em>:27020: registering this node with port 27020
[17:22:24]:<em style='color: #c0392b;'>h3</em>:27019: host added to cluster.
[17:22:24]:<em style='color: #c0392b;'>h3</em>:27019: registering this node with port 27019
[17:22:24]:<em style='color: #1abc9c;'>h2</em>:27019: host added to cluster.
[17:22:24]:<em style='color: #1abc9c;'>h2</em>:27019: registering this node with port 27019
[17:22:24]:<em style='color: #2ecc71;'>h1</em>:27019: host added to cluster.
[17:22:24]:<em style='color: #2ecc71;'>h1</em>:27019: registering this node with port 27019
[17:22:24]:<em style='color: #c0392b;'>h3</em>:27018: host added to cluster.
[17:22:24]:<em style='color: #c0392b;'>h3</em>:27018: registering this node with port 27018
[17:22:24]:<em style='color: #1abc9c;'>h2</em>:27018: host added to cluster.
[17:22:24]:<em style='color: #1abc9c;'>h2</em>:27018: registering this node with port 27018
[17:22:24]:<em style='color: #2ecc71;'>h1</em>:27018: host added to cluster.
[17:22:24]:<em style='color: #2ecc71;'>h1</em>:27018: registering this node with port 27018
[17:22:24]:All 12 hosts are accessible by SSH.
[17:22:22]:Checking ssh/sudo on 12 hosts.
[17:22:22]:Found mongos: ms1:27017,ms2:27017
[17:22:22]:Found db nodes: h1:27018,h2:27018,h3:27018,h1:27019,h2:27019,h3:27019,h1:27020,h2:27020,h3:27020
[17:22:22]:<em style='color: #2ecc71;'>h1</em>:27020: Creating new backup user 'cmon_backup'.
[17:22:22]:<em style='color: #2ecc71;'>h1</em>:27020: Checking for already existing backup user named 'cmon_backup'.
[17:22:22]:<em style='color: #2ecc71;'>h1</em>:27020: Creating cmon backup role.
[17:22:22]:<em style='color: #2ecc71;'>h1</em>:27020: Checking for already existing cmon backup role.
[17:22:22]:Found primary master is <em style='color: #c66211;'>h1:27020</em>.
[17:22:21]:Waiting for a PRIMARY master for replica set shard3.
[17:22:21]:Setting backup user for replicaset 'shard3'.
[17:22:21]:<em style='color: #75599b;'>h3:27020</em>: Verify authentication to MongoDB server.
[17:22:21]:<em style='color: #7415f6;'>h2:27020</em>: Verify authentication to MongoDB server.
[17:22:21]:<em style='color: #c66211;'>h1:27020</em>: Verify authentication to MongoDB server.
[17:22:21]:Checking connection and authentication to relvant nodes.
[17:22:21]:<em style='color: #75599b;'>h3:27020</em>: Verify connection to MongoDB server.
[17:22:21]:<em style='color: #c0392b;'>h3</em>: Detected free disk space: 14321 MB. Total disk space: 25929 MB.
[17:22:21]:<em style='color: #c0392b;'>h3</em>: SSH access (with super-user) is working.
[17:22:21]:<em style='color: #c0392b;'>h3</em>: Looking up the hostname.
[17:22:21]:<em style='color: #c0392b;'>h3</em>: Checking if host belongs to a known cluster.
[17:22:21]:<em style='color: #c0392b;'>h3</em>: Detected OS: centos 6.10 (redhat)
[17:22:21]:<em style='color: #c0392b;'>h3</em>: Detecting the OS.
[17:22:21]:Found new node 'h3:27020' for replica set 'shard3'.
[17:22:21]:<em style='color: #7415f6;'>h2:27020</em>: Verify connection to MongoDB server.
[17:22:21]:<em style='color: #1abc9c;'>h2</em>: Detected free disk space: 16083 MB. Total disk space: 25929 MB.
[17:22:21]:<em style='color: #1abc9c;'>h2</em>: SSH access (with super-user) is working.
[17:22:21]:<em style='color: #1abc9c;'>h2</em>: Looking up the hostname.
[17:22:21]:<em style='color: #1abc9c;'>h2</em>: Checking if host belongs to a known cluster.
[17:22:21]:<em style='color: #1abc9c;'>h2</em>: Detected OS: centos 6.10 (redhat)
[17:22:21]:<em style='color: #1abc9c;'>h2</em>: Detecting the OS.
[17:22:21]:Found new node 'h2:27020' for replica set 'shard3'.
[17:22:21]:Checking configuration of 'shard3' replica set.
[17:22:21]:<em style='color: #1abc9c;'>h2</em>:27019: Creating new backup user 'cmon_backup'.
[17:22:21]:<em style='color: #1abc9c;'>h2</em>:27019: Checking for already existing backup user named 'cmon_backup'.
[17:22:21]:<em style='color: #1abc9c;'>h2</em>:27019: Creating cmon backup role.
[17:22:21]:<em style='color: #1abc9c;'>h2</em>:27019: Checking for already existing cmon backup role.
[17:22:21]:Found primary master is <em style='color: #d35400;'>h2:27019</em>.
[17:22:19]:Waiting for a PRIMARY master for replica set shard2.
[17:22:19]:Setting backup user for replicaset 'shard2'.
[17:22:19]:<em style='color: #f39c12;'>h3:27019</em>: Verify authentication to MongoDB server.
[17:22:19]:<em style='color: #d35400;'>h2:27019</em>: Verify authentication to MongoDB server.
[17:22:19]:<em style='color: #c0392b;'>h1:27019</em>: Verify authentication to MongoDB server.
[17:22:19]:Checking connection and authentication to relvant nodes.
[17:22:19]:<em style='color: #f39c12;'>h3:27019</em>: Verify connection to MongoDB server.
[17:22:19]:<em style='color: #c0392b;'>h3</em>: Detected free disk space: 14321 MB. Total disk space: 25929 MB.
[17:22:19]:<em style='color: #c0392b;'>h3</em>: SSH access (with super-user) is working.
[17:22:19]:<em style='color: #c0392b;'>h3</em>: Looking up the hostname.
[17:22:19]:<em style='color: #c0392b;'>h3</em>: Checking if host belongs to a known cluster.
[17:22:19]:<em style='color: #c0392b;'>h3</em>: Detected OS: centos 6.10 (redhat)
[17:22:19]:<em style='color: #c0392b;'>h3</em>: Detecting the OS.
[17:22:19]:Found new node 'h3:27019' for replica set 'shard2'.
[17:22:19]:<em style='color: #d35400;'>h2:27019</em>: Verify connection to MongoDB server.
[17:22:19]:<em style='color: #1abc9c;'>h2</em>: Detected free disk space: 16083 MB. Total disk space: 25929 MB.
[17:22:19]:<em style='color: #1abc9c;'>h2</em>: SSH access (with super-user) is working.
[17:22:19]:<em style='color: #1abc9c;'>h2</em>: Looking up the hostname.
[17:22:19]:<em style='color: #1abc9c;'>h2</em>: Checking if host belongs to a known cluster.
[17:22:19]:<em style='color: #1abc9c;'>h2</em>: Detected OS: centos 6.10 (redhat)
[17:22:19]:<em style='color: #1abc9c;'>h2</em>: Detecting the OS.
[17:22:19]:Found new node 'h2:27019' for replica set 'shard2'.
[17:22:19]:Checking configuration of 'shard2' replica set.
[17:22:19]:<em style='color: #1abc9c;'>h2</em>:27018: Creating new backup user 'cmon_backup'.
[17:22:19]:<em style='color: #1abc9c;'>h2</em>:27018: Checking for already existing backup user named 'cmon_backup'.
[17:22:19]:<em style='color: #1abc9c;'>h2</em>:27018: Creating cmon backup role.
[17:22:19]:<em style='color: #1abc9c;'>h2</em>:27018: Checking for already existing cmon backup role.
[17:22:19]:Found primary master is <em style='color: #c0392b;'>h2:27018</em>.
[17:22:18]:Waiting for a PRIMARY master for replica set shard1.
[17:22:18]:Setting backup user for replicaset 'shard1'.
[17:22:18]:<em style='color: #d35400;'>h3:27018</em>: Verify authentication to MongoDB server.
[17:22:18]:<em style='color: #c0392b;'>h2:27018</em>: Verify authentication to MongoDB server.
[17:22:18]:<em style='color: #1abc9c;'>h1:27018</em>: Verify authentication to MongoDB server.
[17:22:18]:Checking connection and authentication to relvant nodes.
[17:22:18]:<em style='color: #d35400;'>h3:27018</em>: Verify connection to MongoDB server.
[17:22:18]:<em style='color: #c0392b;'>h3</em>: Detected free disk space: 14321 MB. Total disk space: 25929 MB.
[17:22:17]:<em style='color: #c0392b;'>h3</em>: SSH access (with super-user) is working.
[17:22:17]:<em style='color: #c0392b;'>h3</em>: Looking up the hostname.
[17:22:17]:<em style='color: #c0392b;'>h3</em>: Checking if host belongs to a known cluster.
[17:22:17]:<em style='color: #c0392b;'>h3</em>: Detected OS: centos 6.10 (redhat)
[17:22:17]:<em style='color: #c0392b;'>h3</em>: Detecting the OS.
[17:22:17]:Found new node 'h3:27018' for replica set 'shard1'.
[17:22:17]:<em style='color: #c0392b;'>h2:27018</em>: Verify connection to MongoDB server.
[17:22:17]:<em style='color: #1abc9c;'>h2</em>: Detected free disk space: 16083 MB. Total disk space: 25929 MB.
[17:22:16]:<em style='color: #1abc9c;'>h2</em>: SSH access (with super-user) is working.
[17:22:16]:<em style='color: #1abc9c;'>h2</em>: Looking up the hostname.
[17:22:16]:<em style='color: #1abc9c;'>h2</em>: Checking if host belongs to a known cluster.
[17:22:16]:<em style='color: #1abc9c;'>h2</em>: Detected OS: centos 6.10 (redhat)
[17:22:16]:<em style='color: #1abc9c;'>h2</em>: Detecting the OS.
[17:22:16]:Found new node 'h2:27018' for replica set 'shard1'.
[17:22:16]:Checking configuration of 'shard1' replica set.
[17:22:16]:<em style='color: #da0bf3;'>cf1</em>:4000: Creating new backup user 'cmon_backup'.
[17:22:16]:<em style='color: #da0bf3;'>cf1</em>:4000: Checking for already existing backup user named 'cmon_backup'.
[17:22:16]:<em style='color: #da0bf3;'>cf1</em>:4000: Creating cmon backup role.
[17:22:16]:<em style='color: #da0bf3;'>cf1</em>:4000: Checking for already existing cmon backup role.
[17:22:16]:Found primary master is <em style='color: #f3990b;'>cf1:4000</em>.
[17:22:15]:Waiting for a PRIMARY master for replica set config.
[17:22:15]:Setting backup user for replicaset 'config'.
[17:22:15]:Found config nodes: cf1:4000,cf2:4000
[17:22:15]:<em style='color: #da0bf3;'>cf2:4000</em>: Verify connection to MongoDB server.
[17:22:15]:<em style='color: #877d0f;'>cf2</em>: Detected free disk space: 16499 MB. Total disk space: 25929 MB.
[17:22:15]:<em style='color: #877d0f;'>cf2</em>: SSH access (with super-user) is working.
[17:22:15]:<em style='color: #877d0f;'>cf2</em>: Looking up the hostname.
[17:22:15]:<em style='color: #877d0f;'>cf2</em>: Checking if host belongs to a known cluster.
[17:22:15]:<em style='color: #877d0f;'>cf2</em>: Detected OS: centos 6.10 (redhat)
[17:22:14]:<em style='color: #877d0f;'>cf2</em>: Detecting the OS.
[17:22:14]:Found new node 'cf2:4000' for replica set 'config'.
[17:22:14]:Checking configuration of 'config' replica set.
[17:22:14]:<em style='color: #c66211;'>h1:27020</em>: Verify connection to MongoDB server.
[17:22:14]:<em style='color: #2ecc71;'>h1</em>: Detected free disk space: 16074 MB. Total disk space: 25929 MB.
[17:22:14]:<em style='color: #2ecc71;'>h1</em>: SSH access (with super-user) is working.
[17:22:14]:<em style='color: #2ecc71;'>h1</em>: Looking up the hostname.
[17:22:14]:<em style='color: #2ecc71;'>h1</em>: Checking if host belongs to a known cluster.
[17:22:14]:<em style='color: #2ecc71;'>h1</em>: Detected OS: centos 6.10 (redhat)
[17:22:14]:<em style='color: #2ecc71;'>h1</em>: Detecting the OS.
[17:22:14]:Found shard replica set shard3.
[17:22:14]:<em style='color: #c0392b;'>h1:27019</em>: Verify connection to MongoDB server.
[17:22:14]:<em style='color: #2ecc71;'>h1</em>: Detected free disk space: 16074 MB. Total disk space: 25929 MB.
[17:22:14]:<em style='color: #2ecc71;'>h1</em>: SSH access (with super-user) is working.
[17:22:14]:<em style='color: #2ecc71;'>h1</em>: Looking up the hostname.
[17:22:14]:<em style='color: #2ecc71;'>h1</em>: Checking if host belongs to a known cluster.
[17:22:14]:<em style='color: #2ecc71;'>h1</em>: Detected OS: centos 6.10 (redhat)
[17:22:14]:<em style='color: #2ecc71;'>h1</em>: Detecting the OS.
[17:22:14]:Found shard replica set shard2.
[17:22:14]:<em style='color: #1abc9c;'>h1:27018</em>: Verify connection to MongoDB server.
[17:22:14]:<em style='color: #2ecc71;'>h1</em>: Detected free disk space: 16074 MB. Total disk space: 25929 MB.
[17:22:13]:<em style='color: #2ecc71;'>h1</em>: SSH access (with super-user) is working.
[17:22:13]:<em style='color: #2ecc71;'>h1</em>: Looking up the hostname.
[17:22:13]:<em style='color: #2ecc71;'>h1</em>: Checking if host belongs to a known cluster.
[17:22:13]:<em style='color: #2ecc71;'>h1</em>: Detected OS: centos 6.10 (redhat)
[17:22:13]:<em style='color: #2ecc71;'>h1</em>: Detecting the OS.
[17:22:13]:Found shard replica set shard1.
[17:22:13]:<em style='color: #f3990b;'>cf1:4000</em>: Verify connection to MongoDB server.
[17:22:13]:<em style='color: #da0bf3;'>cf1</em>: Detected free disk space: 20047 MB. Total disk space: 25929 MB.
[17:22:13]:<em style='color: #da0bf3;'>cf1</em>: SSH access (with super-user) is working.
[17:22:13]:<em style='color: #da0bf3;'>cf1</em>: Looking up the hostname.
[17:22:13]:<em style='color: #da0bf3;'>cf1</em>: Checking if host belongs to a known cluster.
[17:22:13]:<em style='color: #da0bf3;'>cf1</em>: Detected OS: centos 6.10 (redhat)
[17:22:12]:<em style='color: #da0bf3;'>cf1</em>: Detecting the OS.
[17:22:12]:Found config replica set config.
[17:22:12]:<em style='color: #2ecc71;'>ms1:27017</em>: got shardMap:
{ "map" : { "cf1:4000" : "config/cf1:4000,cf2:4000", "cf2:4000" : "config/cf1:4000,cf2:4000", "config" : "config/cf1:4000,cf2:4000", "config/cf1:4000,cf2:4000" : "config/cf1:4000,cf2:4000", "h1:27018" : "shard1/h1:27018,h2:27018,h3:27018", "h1:27019" : "shard2/h1:27019,h2:27019,h3:27019", "h1:27020" : "shard3/h1:27020,h2:27020,h3:27020", "h2:27018" : "shard1/h1:27018,h2:27018,h3:27018", "h2:27019" : "shard2/h1:27019,h2:27019,h3:27019", "h2:27020" : "shard3/h1:27020,h2:27020,h3:27020", "h3:27018" : "shard1/h1:27018,h2:27018,h3:27018", "h3:27019" : "shard2/h1:27019,h2:27019,h3:27019", "h3:27020" : "shard3/h1:27020,h2:27020,h3:27020", "shard1" : "shard1/h1:27018,h2:27018,h3:27018", "shard1/h1:27018,h2:27018,h3:27018" : "shard1/h1:27018,h2:27018,h3:27018", "shard2" : "shard2/h1:27019,h2:27019,h3:27019", "shard2/h1:27019,h2:27019,h3:27019" : "shard2/h1:27019,h2:27019,h3:27019", "shard3" : "shard3/h1:27020,h2:27020,h3:27020", "shard3/h1:27020,h2:27020,h3:27020" : "shard3/h1:27020,h2:27020,h3:27020" }, "ok" : 1.0, "operationTime" : { "$timestamp" : { "t" : 1561368216, "i" : 1 } }, "$clusterTime" : { "clusterTime" : { "$timestamp" : { "t" : 1561368216, "i" : 1 } }, "signature" : { "hash" : { "$binary" : "JJ2f3wFuL9F4z80R68PYv31BTIQ=", "$type" : "00" }, "keyId" : 6699399793434689565 } } }
[17:22:12]:<em style='color: #2ecc71;'>ms1:27017</em>: Querying getShardMap.
[17:22:12]:<em style='color: #2ecc71;'>ms1:27017</em>: This is a mongos server, lets collect replica sets.
[17:22:12]:<em style='color: #2ecc71;'>ms1:27017</em>: got shardMap:
{ "map" : { "cf1:4000" : "config/cf1:4000,cf2:4000", "cf2:4000" : "config/cf1:4000,cf2:4000", "config" : "config/cf1:4000,cf2:4000", "config/cf1:4000,cf2:4000" : "config/cf1:4000,cf2:4000", "h1:27018" : "shard1/h1:27018,h2:27018,h3:27018", "h1:27019" : "shard2/h1:27019,h2:27019,h3:27019", "h1:27020" : "shard3/h1:27020,h2:27020,h3:27020", "h2:27018" : "shard1/h1:27018,h2:27018,h3:27018", "h2:27019" : "shard2/h1:27019,h2:27019,h3:27019", "h2:27020" : "shard3/h1:27020,h2:27020,h3:27020", "h3:27018" : "shard1/h1:27018,h2:27018,h3:27018", "h3:27019" : "shard2/h1:27019,h2:27019,h3:27019", "h3:27020" : "shard3/h1:27020,h2:27020,h3:27020", "shard1" : "shard1/h1:27018,h2:27018,h3:27018", "shard1/h1:27018,h2:27018,h3:27018" : "shard1/h1:27018,h2:27018,h3:27018", "shard2" : "shard2/h1:27019,h2:27019,h3:27019", "shard2/h1:27019,h2:27019,h3:27019" : "shard2/h1:27019,h2:27019,h3:27019", "shard3" : "shard3/h1:27020,h2:27020,h3:27020", "shard3/h1:27020,h2:27020,h3:27020" : "shard3/h1:27020,h2:27020,h3:27020" }, "ok" : 1.0, "operationTime" : { "$timestamp" : { "t" : 1561368216, "i" : 1 } }, "$clusterTime" : { "clusterTime" : { "$timestamp" : { "t" : 1561368216, "i" : 1 } }, "signature" : { "hash" : { "$binary" : "JJ2f3wFuL9F4z80R68PYv31BTIQ=", "$type" : "00" }, "keyId" : 6699399793434689565 } } }
[17:22:12]:<em style='color: #2ecc71;'>ms1:27017</em>: Querying getShardMap.
[17:22:12]:<em style='color: #2ecc71;'>ms1</em>:27017: mongod basedir: <em style='color: #0b33b5;'>/usr/</em>
[17:22:12]:<em style='color: #2ecc71;'>ms1:27017</em>: Checking the MongoDB configuration.
[17:22:12]:No backup user name was specified, using default: 'cmon_backup'.
[17:22:12]:<em style='color: #2ecc71;'>ms1:27017</em>: Verify connection to MongoDB server.
[17:22:12]:<em style='color: #2ecc71;'>ms1</em>: Detected free disk space: 20047 MB. Total disk space: 25929 MB.
[17:22:12]:<em style='color: #2ecc71;'>ms1</em>: SSH access (with super-user) is working.
[17:22:11]:<em style='color: #2ecc71;'>ms1</em>: Looking up the hostname.
[17:22:11]:<em style='color: #2ecc71;'>ms1</em>: Checking if host belongs to a known cluster.
[17:22:11]:<em style='color: #2ecc71;'>ms1</em>: detected mongodb version: 4.0, used input data:
mongodb-org.x86_64 4.0.10-1.el6 @mongodb-org-4.0
[17:22:11]:<em style='color: #2ecc71;'>ms1</em>: Version of package <strong style='color: #110679;'>*mongodb*</strong> is 4.0.
[17:22:11]:<em style='color: #2ecc71;'>ms1</em>: Verifying the <strong style='color: #110679;'>*mongodb*</strong> installation.
[17:22:11]:<em style='color: #2ecc71;'>ms1</em>: Detected OS: centos 6.10 (redhat)
[17:22:11]:<em style='color: #2ecc71;'>ms1</em>: Detecting the OS.
[17:22:11]:Verifying job parameters.
[17:22:11]:Generating configuration for cluster <strong style='color: #59a449;'>MongoDB Sharded Cluster</strong>.
[17:22:10]:Adding cluster with type '<strong style='color: #59a449;'>mongodb</strong>', vendor '<strong style='color: #59a449;'>10gen</strong>'.
Job spec: {
"command": "add_cluster",
"group_id": 1,
"group_name": "admins",
"job_data":
{
"api_id": 1,
"cluster_name": "MongoDB Sharded Cluster",
"cluster_type": "mongodb",
"config_template": "mongodb.conf.percona",
"datadir": "/var/lib/mongodb",
"db_password": "admin123",
"db_user": "root",
"disable_firewall": true,
"disable_selinux": true,
"enable_uninstall": true,
"generate_token": true,
"install_software": true,
"mongodb_authdb": "admin",
"mongos_conf_template": "mongos.conf.org",
"mongos_servers": [
{
"hostname": "ms1",
"hostname_data": "ms1",
"hostname_internal": "",
"port": "27017"
},
{
"hostname": "ms2",
"hostname_data": "ms2",
"hostname_internal": "",
"port": "27017"
} ],
"port": "27017",
"ssh_keyfile": "/root/.ssh/id_rsa",
"ssh_port": "22",
"ssh_user": "root",
"sudo_password": "",
"user_id": 1,
"vendor": "10gen",
"version": "4.0"
},
"user_id": 1,
"user_name": "dep.si.dm@incrte.com"
}
Please sign in to leave a comment.
Comments
1 comment