Articles in this section

See more

Getting Started: ClusterControl and Chef Solo

Avatar
Alex
  • Updated
Follow

Chef

Chef is a Ruby based open source systems integration framework for managing your infrastructure. By using 'recipes' which describes what state resources (applications, services, files etc) on your servers should be in, you can easily make sure the servers are always setup and running exactly the way as you want them to be.

Chef comes in two flavours, a client/server setup and a standalone variant. You can manage your own Chef servers, use a hosting service with Opscode Inc or for managing a single server use Chef-solo.

ClusterControl

ClusterControl enables you to manage, monitor and scale out your database deployments. The core components comprises of a controller and one or more agents which provides a single coherent view into your database cluster.

Combining Chef and ClusterControl allows you to quickly provision and setup your existing or new database infrastructure with ClusterControl in order to manage, monitor and easily scale out your database cluster.

 

Single MySQL Master

In the next following sections we will guide you through provisioning and monitoring a single MySQL server using ClusterControl and Chef Solo.

We start first by installing ClusterControl to a server that will serve as our management server which will host ClusterControl's web application and a controller.

Then on a second server we install a MySQL server that we want to manage and monitor together with a ClusterControl agent.
The distribution used in the tutorial is Ubuntu (11.10) and we are assuming you are logged in as root.

ClusterControl Controller Node

1) Install Chef-Solo

We need the latest Chef version which is 0.10.8.

node1:~$ apt-cache show chef
Package: chef
Priority: extra
Section: universe/ruby
Installed-Size: 204
Maintainer: Ubuntu Developers <ubuntu-devel-discuss@lists.ubuntu.com>
Original-Maintainer: Joshua Timberman <joshua@opscode.com>
Architecture: all
Version: 0.8.16-4.2
Depends: debconf (>= 0.5) | debconf-2.0, ruby1.8, liberubis-ruby1.8, libjson-ruby1.8, libextlib-ruby1.8 (>= 0.9.13), ohai (>= 0.4.0), libchef-ruby1.8 (= 0.8.16-4.2), libopenssl-ruby1.8, libmixlib-authentication-ruby1.8 (>= 1.1.0), ucf
Recommends: irb1.8
Filename: pool/universe/c/chef/chef_0.8.16-4.2_all.deb
Size: 20154
MD5sum: 07e3fbf6b63ba25666a000deec721607
SHA1: b36a00739f259bfd448c70211c0b3f90f30e7ea8
SHA256: 85809a9e5a823f1ca265c5bffd7ff93343c49c3cb7b38970f5ec789ee91ebf32
Description: system integration framework - client and solo binaries
Homepage: http://wiki.opscode.com/display/chef/Home
Description-md5: e270d0463c20b049b7c7125b65c51216
Bugs: https://bugs.launchpad.net/ubuntu/+filebug
Origin: Ubuntu

OK, so we need another way to install the latest and greatest Chef version.

If your distribution is not Debian/Ubuntu based then please following the installation instructions for other distributions and OSes here:
http://wiki.opscode.com/display/chef/Installing+Chef+Client+and+Chef+Solo

We are going to add Opscode's APT repository. Instructions also available here:
http://wiki.opscode.com/display/chef/Installing+Chef+Client+on+Ubuntu+or+Debian

node1:~$ echo "deb http://apt.opscode.com/ `lsb_release -cs`-0.10 main" | sudo tee /etc/apt/sources.list.d/opscode.list
deb http://apt.opscode.com/ oneiric-0.10 main

Then add Opscode's GPG key.

node1:~$ mkdir -p /etc/apt/trusted.gpg.d
node1:~$ gpg --keyserver keys.gnupg.net --recv-keys 83EF826A
gpg: directory `/root/.gnupg' created
gpg: new configuration file `/root/.gnupg/gpg.conf' created
gpg: WARNING: options in `/root/.gnupg/gpg.conf' are not yet active during this run
gpg: keyring `/root/.gnupg/secring.gpg' created
gpg: keyring `/root/.gnupg/pubring.gpg' created
gpg: requesting key 83EF826A from hkp server keys.gnupg.net
gpg: /root/.gnupg/trustdb.gpg: trustdb created
gpg: key 83EF826A: public key "Opscode Packages <packages@opscode.com>" imported
gpg: Total number processed: 1
gpg: imported: 1
node1:~$ gpg --export packages@opscode.com | sudo tee /etc/apt/trusted.gpg.d/opscode-keyring.gpg > /dev/null

Update our apt repo.

node1:~$ apt-get update

Install Opscode's APT key

node1:~$ apt-get install opscode-keyring # permanent upgradeable keyring

To ensure you are using the latest versions of libraries that Chef depends on.

node1:~$ apt-get upgrade

Then finally install Chef-Solo. When prompted for Chef Server URL, enter 'none' for Chef-Solo.

node1:~$ apt-get install chef
node1:~$ which chef-solo
/usr/bin/chef-solo

2) Install ClusterControl Chef Cookbooks

We have created a github repository for our cookbooks here: https://github.com/severalnines/S9s_cookbooks
Brief readme here: https://github.com/severalnines/S9s_cookbooks/blob/master/cmon/README.md

You can either download a zip/tar.gz file of the repository to get our Chef recipies or just clone the repo with git.

(node1:~$ apt-get install git)
node1:~$ git clone https://github.com/severalnines/S9s_cookbooks.git
Cloning into S9s_cookbooks...
remote: Counting objects: 97, done.
remote: Compressing objects: 100% (45/45), done.
remote: Total 97 (delta 36), reused 90 (delta 29)
Unpacking objects: 100% (97/97), done.

Ok, assuming the repo has been downloaded/cloned, you should see the following files in repo.

node1:~$ ls S9s_cookbooks
cmon      # our ClusterControl cookbook with recipes for the controller and agent
solo.rb   # Chef-solo configuration file
solo.json # Chef-Solo attributes

Next, let's edit a data bag item which specifies ClusterControl's controller IP address and set the host_ipaddress key to 192.168.122.3 (node1).

node1:~$ vi S9s_cookbooks/cmon/databags/controller/config.json
config.json should looks something like below
{
 "id": "config",
 "controller_host_ipaddress": "192.168.122.3",
 "type": "replication",
 "mode": "controller",
 "cmon_tarball_x86_64": "cmon-1.1.27-64bit-glibc23-mc70.tar.gz",
 "cmon_tarball_i686": "cmon-1.1.27-32bit-glibc23-mc70.tar.gz",
 "cmon_tarball_i386": "cmon-1.1.27-32bit-glibc23-mc70.tar.gz",
 "agent_hosts": [
   "ip1",
   "ip2"
  ]
 }

The 'mode' key is set to 'agent' for agents and 'dual' when a controller acts both as a controller and agent in case you only have a single server available.

While we are at it let's add some agent(s) hosts. Later we'll install an agent on node2 with IP address 192.168.122.4

The agent_hosts key should  should look something like below

{
  ...
  "agent_hosts": [
    "192.168.122.4"
    ]
}

Specifying three agents in the hosts.json file would look something like this.

{
  ...
  "agent_hosts": [
    "192.168.122.4",
    "192.168.122.6",
    "192.168.122.8"
    ]
}

Next check the default attributes, S9s_cookbooks/cmon/default/attributes.rb
Important settings to verify is the MySQL installation paths and MySQL root password. The MySQL installation paths are set depending on the distribution used but it's good to check that it looks accurate. 

Next include which recipes to run by editing the default.rb reciepe which is the 

recipe that is selected as default in solo.json's runlist.

In this tutorial we're installing on a freshly created Ubuntu 11.10 server with few any additional packages installed. For example there are no apache or php packages.

So we need to include all the package dependencies as well. If you already have apache2,
php5, rrdtool and mysql installed you would only need to include the controller recipe,  cmon::controller and cmon::web.

node1:~$ vi S9s_cookbooks/cmon/recipes/default.rb

You should end up with something like this.

# cmon::controller_packages
# installs 
# rrdtool mysql mysql-server (rhel/centos)
# rrdtool mysql-server (debian)
#
# cmon::controller
# installs the controller
include_recipe "cmon::controller_packages"
include_recipe "cmon::controller"
# cmon::web_packages
# installs
# httpd php php-mysql php-gd (rhel/centos)
# apache2 php5-mysql php5-gd (debian/ubuntu)
#
# cmon::web_packages
# installs ClusterControl web application (php)
include_recipe "cmon::web_packages"
include_recipe "cmon::web"


That's the last edit! Let's install ClusterControl's controller.

3) Install ClusterControl Controller

Let's run Chef Solo.

node1:~$ cd S9s_Cookbooks
node1:~$ chef-solo -c solo.rb -j solo.json

Verify that the web application is also working by going to http://<your controller host IP address>/cmon

You might need to go into 'Global Settings' and change the ClusterControl's database host IP address if it's set to 127.0.0.1/localhost.

ClusterControl Agent Node

1) Install MySQL

If you already have MySQL installed then please take a note of your root or a user with grant access. We need to grant access to the MySQL server for the controller and agent.

Install a MySQL server.

node1:~$ apt-get -y install mysql-server

We're using 'password' as root's password in our example. If you didn't get a password prompt during the installation you can secure your database by doing the following:

node1:~$ mysql_secure_installation
NOTE: RUNNING ALL PARTS OF THIS SCRIPT IS RECOMMENDED FOR ALL MySQL
SERVERS IN PRODUCTION USE! PLEASE READ EACH STEP CAREFULLY!
In order to log into MySQL to secure it, we'll need the current
password for the root user. If you've just installed MySQL, and
you haven't set the root password yet, the password will be blank,
so you should just press enter here.
Enter current password for root (enter for none):

and enter 'password' as the password.

Test if you can access the MySQL server.

node1:~$ mysql -uroot -ppassword
Welcome to the MySQL monitor. Commands end with ; or \g.
Your MySQL connection id is 39
Server version: 5.1.58-1ubuntu1 (Ubuntu)
Copyright (c) 2000, 2010, Oracle and/or its affiliates. All rights reserved.
This software comes with ABSOLUTELY NO WARRANTY. This is free software,
and you are welcome to modify and redistribute it under the GPL v2 license
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
mysql>

2) Install Chef Solo

Copy and paste this into a shell script and use it to bootstrap the latest Chef version on your servers (run as root).

#!/usr/bin/env bash
echo "deb http://apt.opscode.com/ `lsb_release -cs`-0.10 main" | sudo tee /etc/apt/sources.list.d/opscode.list
mkdir -p /etc/apt/trusted.gpg.d
mkdir -p /etc/apt/trusted.gpg.d
gpg --keyserver keys.gnupg.net --recv-keys 83EF826A
gpg --export packages@opscode.com | sudo tee /etc/apt/trusted.gpg.d/opscode-keyring.gpg > /dev/null
apt-get -y update
apt-get -y install opscode-keyring
apt-get -y upgrade
apt-get -y install chef

Remember to enter 'none' when prompted for the Chef URL.

3) Install ClusterControl Chef Cookbooks

Simply copy over node1's cookbook repository to node2 which where we will install the agent.

node2:~$ scp -r root@node1:/root/S9s_cookbooks .

Then edit the default recipe to include only the ClusterControl agent recipes while commenting out the controller and web recipes.

node2:~$ vi S9s_cookbooks/cmon/recipes/default.rb
You should end up with something like below

# cmon::agent_packages
# installs 
# psmisc libaio (rhel/centos)
# psmisc libaio1 (debian/ubuntu)
#
# cmon::agent
# installs the agent
include_recipe "cmon::agent_packages"
include_recipe "cmon::agent"

3) Install ClusterControl Agent

Finally run Chef-solo.

node1:~$ cd S9s_Cookbooks
node1:~$ chef-solo -c solo.rb -j solo.json

Verify that the web application is also working by going to http://<your controller host IP address>/cmon

You might need to go into 'Global Settings' and change the ClusterControl's database host IP address if it's set to 127.0.0.1/localhost.

 

Troubleshooting

Q: ClusterControl controller/agent cannot connect to the MySQL server?
A: First check /etc/cmon.cnf and /var/log/cmon.log that you are using the correct host IP address, user and password.

Second check your MySQL configuration file my.cnf usually in /etc/mysql/my.cnf or /etc/my.cnf
If you have 'bind-address = 127.0.0.1' then remote clients cannot connect to your MySQL server. Either set bind-address to point to a remotely reachable address or remove it.

Q: Changing ClusterControl's DB host IP address does not take effect in 'Global Settings'?
A: Try restart the apache server with

$ sudo apachectl restart
or
$ service apache2 restart
or
$ /etc/init.d/apache2 restart

Q: What type of cluster are the controller and agent initially set to?
A: replication cluster

Q: In the web application my cluster is in an unknown state?
A: Yes, that usually shows up if you have a single instance of the MySQL server that does not have server-id nor log_bin parameters enabled.

As default the controller and agent's the cluster type is set to replication.

Q: What are the cluster types?
A: We support currently three types:

* replication
This type manages and monitors a Master to standby readonly Master setup (with slaves).

If you add another MySQL server to your current single server "cluster" it will become a second standby readonly Master, i.e., we will setup a one way semi-synchronous replication from your first MySQL server to the second.

* MySQL Cluster
MySQL Cluster deployments with SQL and Data Nodes, i.e., the NDB storage engine.

* Galera
Galera synchronous multi-master MySQL server cluster.

Q: What DB privileges are set?
A:
On the controller:

-- controller
GRANT INSERT,UPDATE,DELETE,SELECT ON cmon.* TO 'cmon'@'127.0.0.1' IDENTIFIED BY 'cmon';
GRANT INSERT,UPDATE,DELETE,SELECT ON cmon.* TO 'cmon'@'<host IP adress>' IDENTIFIED BY 'cmon';
GRANT SUPER on *.* TO 'cmon'@'127.0.0.1' IDENTIFIED BY 'cmon';
GRANT SUPER on *.* TO 'cmon'@'<host IP address>' IDENTIFIED BY 'cmon';

-- For each agent:
GRANT INSERT,UPDATE,DELETE,SELECT ON cmon.* TO 'cmon'@'<agent IP address>' IDENTIFIED BY 'cmon';
GRANT SUPER *.* TO 'cmon'@'<agent IP address>' IDENTIFIED BY 'cmon';

On the agent

GRANT INSERT,UPDATE,DELETE,SELECT ON cmon.* TO 'cmon'@'127.0.0.1' IDENTIFIED BY 'cmon';
GRANT INSERT,UPDATE,DELETE,SELECT ON cmon.* TO 'cmon'@'<controller IP address>' IDENTIFIED BY 'cmon';
GRANT SUPER, RELOAD, REPLICATION CLIENT ON *.* TO 'cmon'@'127.0.0.1' IDENTIFIED BY 'cmon';
GRANT SUPER, RELOAD, REPLICATION CLIENT ON *.* TO 'cmon'@'<controller IP address>' IDENTIFIED BY 'cmon';
GRANT SELECT ON mysql.* TO 'cmon'@'<controller IP address>' IDENTIFIED BY 'cmon';
GRANT SELECT ON ndbinfo.* TO 'cmon'@'<controller IP address>' IDENTIFIED BY 'cmon'; 

Related articles

Comments

0 comments

Please sign in to leave a comment.

Powered by Zendesk