Upgrading the cluster with apt-get update/upgrade or yum

Comments

10 comments

  • Avatar
    Graham Green

    After upgrading when I go to the dashboard I get "Invalid query: Unknown column 'ignored' in 'where clause'" underneath the menu at the top of the page. The rest of the page is blank.

  • Avatar
    Brian Gregg

    Johan,

    Can you explain step #3 in the above sequences a little bit more?  I'm not sure I understand completely what 'update-rc.d -f mysql remove  && update-rc.d -n mysql stop 20 0 1 6' means.  I've looked at the /etc/rc.d./mysql script and I do not see a 'remove' option so this has thrown me off a bit.  I've ignored this step and have upgraded three nodes in a cluster, however each time I upgraded a  node  it required a recovery process.  I simply waited until everything was "green" before I continued on.

    Please advise if possible.

    Thanks in advance.

  • Avatar
    Johan

    Graham, what did you upgrade? ClusterControl?  In that case did you apply the new schema?

  • Avatar
    Johan

    Brian,

    The instructions have been updated. The idea of the update-rc.d stuff (for Ubuntu, Centos only), is to shutdown the mysql server gracefully, on node reboot, but to not start it automatically (we let the CMON controller do that part). You don't have to do this step, but if you do, then the node will start to recover slight faster than if you don't.

    But yes, otherwise, take one node at a time, let it be green then proceed to the next one.

    When doing manual operations on a node that affects the mysql server (e.g upgrading it which has as a side effect that the mysql server will be shutdown), we do recommend that you shutdown the controller, so it does not try to recover it before you have finished upgrading the packages on the node that you need to have updated. The controller does not know that your are doing an upgrade. It just sees that the node has timed out and will start recovery, which may be or not may be what you want in that situation.

    If shutting down the controller is needed or not really depends on the system, the database size, the load on the server when it is being upgraded etc etc, so we take a carte blanche and just recommend you shutdown the controller, do the upgrade, restart the controller, let the controller recover the node  and then take the next one.

    I hope this clarifies somewhat!

    Good luck,

    Johan

  • Avatar
    Niko

    Hey,

    And how updates are made automatically in Enterprise version?

    Regards,

    Niko

  • Avatar
    Ashraf Sharif

    Hi Niko,

    Kindly review following blog post for details:

    http://www.severalnines.com/blog/patch-updates-and-new-version-upgrades-your-database-clusters

    Regards,

    Ashraf

  • Avatar
    Olivia

    Hi, is this information still valid 5 years later?

    So first update ClusterControl and then run those commands on the Data nodes, then SQL/API nodes, then Mgmt?

     

    Thanks in advance,

    Olivia

  • Avatar
    Ashraf Sharif

    Hi Olivia,

    The steps are still relevant to date. However, instead of shutting down CMON service when performing database upgrade, you could also schedule maintenance mode for all nodes in the cluster so ClusterControl won't try to recover the nodes while under maintenance.

    Then, proceed with upgrading data nodes, SQL/API nodes and then Management nodes. Do perform this exercise during the lowest peak hours.

    Regards,
    Ashraf

  • Avatar
    Johan

    Hi,

    I would, in case of MySQL Cluster to do it in this order:

    1) upgrade the management servers first (this way you know the config.ini still works with the new version of MySQL Cluster).

    2) upgrade the data nodes in a rolling fashion (one at a time).

    3) upgrade the mysql server / sql nodes in a rolling fashion 

    Also, take a backup before upgrading.

    BR

    johan

  • Avatar
    Olivia

    Thank you both so much for your replies!

    Is it OK to add the official MySQL Yum repository to our nodes and update the mysql version too?

     

    Regards,

    Olivia

Please sign in to leave a comment.

Powered by Zendesk