SOLVED: RPC endpoint unavailable Cluster Control "Cluster " is not running

Came across an issue regarding not being able to contact my own RPC endpoint.  It suggested that i check the rpc_key settings in /etc/cmon.conf which were fine and matched the api key

Here is how I fixed it:

Check cmon_*.log for error relating to the hostname of your cluster control resolving to 127.0.0.1

cat /var/log/cmon_* | grep -iP 'error(.*)127.0.0.1'

If you see any errors listed you have an /etc/hosts entry which is most likely overriding your DNS A record for cluster control

Remove the following from /etc/hosts where your.hostname.com is the cluster control hostname fqdn

127.0.0.1     your.hostname.com

Add the following to /etc/hosts where x.x.x.x is the public IP of cluster control's panel, your.hostname.com is the cluster control hostname fqdn

echo "x.x.x.x    your.hostname.com" >> /etc/hosts

Add the following to the top of /etc/default/cmon

RPC_BIND_ADDRESSES="0.0.0.0"

Full /etc/default/cmon should look something like this

# New events client http callback as of v1.4.2!
RPC_BIND_ADDRESSES="0.0.0.0"
EVENTS_CLIENT="http://127.0.0.1:9510"

Restart cmon

service cmon restart