Deployment scripts for Galera 2.2 - when?
Just one quick question: when do you (severalnines) plan to release deployment scripts for Galera 2.2?
Percona 5.5.28 is available, I`m waiting for your move :)
I have try to install and some problems arrived.
I have direct root access to all node from CC (no SUDO) but deploy script try to 'gpg -a --export CD2EFD2A | sudo apt-key add -' which failed. Manual scripts modify solved the problem.
Also Percona repo addon was incorrect - lack of Debian version info in deploy script (in my case lsb-release was not installed, under fresh, latest Debian6. Installation of ' lsb_release' should be added to deploy script.
After all it stuck at: cannot logon into node2. i`m guessing it` because this: wsrep_provider_version | 2.2(r115) (there was discussion on codership mailing list at 16/11/2012).
I hope it`s useful.
Update: Percona`s apt repo for Debian6 is broken! It install old r115 revision on Galera 2.2. New from Percona webpage (r137).
I give up. Even with this, Galera deployment failed. Command that stop deploy.sh is:
'LD_LIBRARY_PATH=/usr/lib/:/usr/lib//mysql /usr/bin//mysql --defaults-file=/etc/mysql///my.cnf -uroot -p'password' -e "SET WSREP_ON=0;SET SQL_LOG_BIN=0; GRANT ALL ON *.* TO 'root'@'192.168.2.20' IDENTIFIED BY 'password'"'
Manual check: can`t logon using 'password' password on that node. SST is broken? No idea...
Thanks for running through this. I will immediately disable Percona then. Please use Codership.
Since the GRANTs failed, then the SST was broken.
Also, many thanks for the 'sudo' and the 'lsb_release' part, will check it out asap.
I will let you know here what happens on my side. Don't expect any miracles today however.
In my opinion Percona version should me more 'stable' as it`s more recognize brand on the MySQL market. I know they are relatively new on Galera market but still.
Can you tell me, how can I clean broken installation? Remove all Percona stuff and clean system up. Preparing from scratch four machines is not an easy process.]
Anyway, your scripts are very efficient and easy to modify. We are using Galera 1.0 and 2.1 in dev environment and they rock! Good job.
I am happy to hear you like them.
If you take the 2.2 using the codership, then all previous MySQL and percona packages will be removed from the hosts you are using. E.g, the following is cleaned out:
dpkg -P MySQL*
dpkg -P percona*
dpkg -P mysql*
Let me know if it helps.
Yesterday I could reproduce your problems (with Percona), but today, they have disappeared. I don't know what's going on, but yesterday it seems that wsrep_sst_rsync and other files were missing/not installed. Today they are.
I have also added an extra step in bootstrap.sh, which calls a function defined in .s9s/help_func called 'mysqld_synched'.
mysqld_synched waits for the node to become synched before proceeding with the installation. I found out that the Percona etc/init.d scripts for starting mysql seems to return earlier than the mysql initd scripts that comes from Codership (which waits to return until the mysqld is in stated Synched). Hence, since they would return early, and the node not being in Synched, then all sorts of strange things will happen.
Anyways, it would be great if you can retry the Percona installation.
Here is what the output is from waiting on Synched:
192.168.56.104: Executing '/etc/init.d/mysql start --wsrep-cluster-address=gcomm://192.168.56.102:4567' [ok]
192.168.56.104: Executing 'sync' [ok]
Waiting for 192.168.56.104 to become Synched
Retry 1 / 600. Node is in state: 2002
Retry 2 / 600. Node is in state: 2002
Retry 3 / 600. Node is in state: 2002
Retry 4 / 600. Node is in state: 2002
Retry 5 / 600. Node is in state: 2002
Retry 6 / 600. Node is in state: 2002
Retry 7 / 600. Node is in state: 2002
Retry 8 / 600. Node is in state: 2002
Retry 9 / 600. Node is in state: 2002
Retry 10 / 600. Node is in state: 2002
Retry 11 / 600. Node is in state: Synced
Node reached state: Synced
Yes, I can try it know. Yesterday I have install Codership version. All went smooth. The only thing required my manual intervention was uninstall of Percona server (due to installation mysql on separated, XFS filesystem: more here http://blog.secaserver.com/2012/05/mysql-live-backup-lvm-snapshots/ - very useful). I don`t know why but Percona uninstall required to delete /var/lib/mysql, which was on mounted fs. So maunal uninstall on all nodes via umount fs first, then Codership version deploy in time of drinking one cup of coffee. One problem I can see no info in column 'Host stats" for first node (the one used as donor for the rest). How to solved it?
Regards new feature in bootstrap - I have such a errors in my log too. Once scirpt was retry in case of long start of mysql, it take sometimes 3-5sec to report online. Double checked via manula restart od the node.
Do you thing that Perocona fix repo? How is your wsrep_provider_version reported (r137 or r115)?
This time deploy went smooth. No manual intervention required. But the problem with lack of 'Host stats' for Node1 is thil there. How to remove it?
Also please notice which version of Galera is installed! Fragment of the 'dpkg --list' form node1
ii percona-xtrabackup 2.0.4-487.squeeze Open source backup tool for InnoDB and XtraDB
ii percona-xtradb-cluster-client-5.5 5.5.28-23.7-369.squeeze Percona Server database client binaries
ii percona-xtradb-cluster-common-5.5 5.5.28-23.7-369.squeeze Percona Server database common files (e.g. /etc/mysql/my.cnf)
ii percona-xtradb-cluster-galera-2.x 115.squeeze Galera components of Percona XtraDB Cluster
ii percona-xtradb-cluster-server-5.5 5.5.28-23.7-369.squeeze Percona Server database server binaries
List of file on Percona download webpage: http://www.percona.com/downloads/Percona-XtraDB-Cluster/LATEST/deb/squeeze/x86_64/
Combine with thread on Codership mailing list, is this deployment ready for production? Have you got any news from Percona? Maybe there is a bug? I don`t fill comfortable with this resync of apt repo :(
After all this test my conclusion is: I will stay with Codership version but I want to switch SST/IST from rsync to xdtabackup. How to do that? Or it`s not possible using Codership mysql version.
- xtrabackup can be used. However, the wsrep_sst_xtrabackup script is needed. I will locate it for you and post it here. Then you can install it in /usr/bin/ or /usr/local/mysql/bin (MYSQL_BASEDIR/bin).
Then you need to change in my.cnf on all hosts to set the new wsrep_sst_method, and you should also change in s9s-galera-2.2.0/mysql/config/my.cnf .
- you also asked for server stats while a node was a DONOR. I have pushed a fix for this but you can change it if you want:
www/cmon/lib/mysql/galera/galeraimpl.php you can change (around line 331):
- if(count($perf_stats) && $status==0)
+ if(count($perf_stats) && ($status==0 || $status==14))
About Percona, I have not had time to dig into it more, but it appears to be working now (Ubuntu 12.04 and Debian 6.0.latest)
You can get wsrep_sst_xtrabackup:
The place it in either:
/usr/bin/ or /usr/local/mysql/bin depending where your MYSQL_BASEDIR is (well, MYSQL_BASEDIR/bin).
Naturally you need to install xtrabackup on all nodes too. You can install it with (requires apt /yum repo access):
s9s_backup --install /etc/mysql/my.cnf
s9s_backup --install /etc//my.cnf
I have tested this on a cluster with a couple of gigs of data and it works as expected. If it is faster than rsync i don't know yet.
Thx for the script. The goal is not speed on first place but non-blocking operation on donor node during sync. As I understood it :) Rsync (current solution) push donor node into read-only mode I want to avoid it (and be still able to load-balance mysql access for rw operations between two nodes). HAproxy is doing good job here. Very good job.
Please sign in to leave a comment.