CMON does not pull Network, CPU, Disk stat from Nodes

March 05, 2014 12:54

I have installed cmon in a CentOS 6.4 x64 box using auto installer for my existing 3 node galera cluster. However i can't see any stats relating to Disk , CPU and Network. However when i check from nodes section i can see every details of cmon node but not the galera nodes. I even reinstalled cmon few time from scratch. Still problem persist.

Any help would be much appreciated.

Screenshot from 2014-03-05 18:23:46.png

Comments

13 comments

Johan

March 05, 2014 13:02
Hi,

What version of CMON are you using?

rpm -qa |grep cmon

BR

johan
0

Comment actions Permalink
Anushka Dissanayaka

March 05, 2014 13:04
Hello Johan,

Its cmon-controller-1.2.5-137.x86_64

Thanks.
0

Comment actions Permalink
Johan

March 05, 2014 15:17
Hi,

And can you from the controller ssh without passwords to the other nodes?
The best would be to the see /var/log/cmon.log from the controller.

If you want to put it here or if you want to create a support ticket for it.

BR
johan
0

Comment actions Permalink
Anushka Dissanayaka

March 05, 2014 15:32
Hi,

Yes, I can ssh into all nodes. Log file attached.
0

Comment actions Permalink
Ashraf Sharif

March 05, 2014 17:13
Hi,

Can you make sure that in ClusterControl's /etc/cmon.cnf contains following line:

agentless=1

If it does not exist, please add it and restart cmon by using "service cmon restart".

Regards,

Ashraf
0

Comment actions Permalink
Anushka Dissanayaka

March 06, 2014 05:10
Hello Ashraf,

Thanks It worked. Is that due to a bug in installer? Because i am certain that i used agentless method.
0

Comment actions Permalink
Ashraf Sharif

March 06, 2014 05:41
Hi Anushka,

I need to double check on this. May I know what is your environment (DB cluster type/vendor, OS distribution, virtualization platform etc)? Did you use bootstrap script from http://www.severalnines.com/downloads/cmon/cc-bootstrap.tar.gz?

Regards,

Ashraf
0

Comment actions Permalink
Anushka Dissanayaka

March 06, 2014 07:59
Hi Ashraf,

Below are the information you requested.

Cluster Type : Galera(PXC)

OS : CentOS 6.4 x64

Platform : On VMWare and EC2 separate clusters

And yes i downloaded the script from the location you mentioned.

Also we have another cluster setup on DigitalOcean and cmon deployed with agentless=1 and all worked fine. Even though i used same method for installing, on EC2 and VMware agentless=1 is not presented.
0

Comment actions Permalink
Chris Church

February 16, 2015 03:42
I'm having this same problem on ubuntu 12.04, also with PXC. This cluster was configured in the last few weeks with the latest version on severalnines.com.

Interestingly, I can see the top for each node, just not cpu/disk/etc. Also, the stats don't work for the controller either.

Yes, passwordless SSH works and agentless=1.

This was set up using the severalnines "install over an existing cluster" option (the nodes were created independently of the cluster monitoring server).

The mysql stats all work fine, top shows up, and the cluster monitor will restart mysql if I manually stop it on a node.

Was this problem resolved in March of last year?
0

Comment actions Permalink
Alex

February 16, 2015 06:20
Hi

If you are on 1.2.8 or older then can you please try to upgrade to our latest version 1.2.9 and see if that issue persists.

Follow the instructions in this post, http://support.severalnines.com/entries/21095371-CMON-1-2-9-released-upgrade-instructions
0

Comment actions Permalink
Johan

February 16, 2015 08:27
Hi Chris,

Are you running on bare metal, or on VMs? Do you have a /proc on the nodes (some virtualizers do not) and do you have e.g /proc/diskstats ?

BR

johan
0

Comment actions Permalink
Chris Church

February 16, 2015 13:52
I'm running on VMs at a hosted datacenter. As the cmon user, /proc/diskstats (and meminfo and cpuinfo) are all readable on the nodes.

I'm running 1.2.8. I can try upgrading.
0

Comment actions Permalink
Chris Church

February 16, 2015 22:17
Looking at the fairly jenky upgrade process to go from 1.2.8 to 1.2.9, that's not something I can turn around quickly. I was hoping that there was something in configuration or logs that could be relevant, though I see nothing in cmon.log.

For what it's worth, I can see the following process periodically running on the cluster monitoring server:

/bin/bash -c 'for f in /proc/[0-9][0-9][0-9]*/stat /proc/[0-9][0-9][0-9]*/statm /proc/[0-9][0-9][0-9]*/status /proc/uptime; do echo %f$f; cat $f | sed -e "s/%/%%/g"; done 2>/dev/null'

Running this command as the cmon user produces a giant list of data.
0

Comment actions Permalink

Please sign in to leave a comment.

Comments

Didn't find what you were looking for?