This is the documentation for Cloudera Enterprise 5.8.x. Documentation for other versions is available at Cloudera Documentation.

Performing a Rolling Upgrade to CDH 5.6

Minimum Required Role: Cluster Administrator (also provided by Full Administrator)

  Important: This feature is available only with a Cloudera Enterprise license. It is not available in Cloudera Express. For information on Cloudera Enterprise licenses, see Managing Licenses.

The rolling upgrade feature takes advantage of parcels and the HDFS high availability to enable you to upgrade your cluster software and restart the upgraded services without taking the entire cluster down. You must have HDFS high availability enabled to perform a rolling upgrade.

This page described how to perform a rolling upgrade between maintenance and minor versions of CDH 5, except Beta versions. For rolling upgrade between CDH 4 versions, see Performing a Rolling Upgrade on a CDH 4 Cluster.

It is not possible to perform a rolling upgrade from CDH 4 to CDH 5 because of incompatibilities between the two major versions. Instead, follow the instructions for a full upgrade at Upgrading from CDH 4 to CDH 5 Parcels.

The steps to perform a rolling upgrade of a cluster to CDH 5.6 are as follows:

  1. Before You Begin
  2. Ensure High Availability is Enabled
  3. Back up HDFS Metadata
  4. Run the Upgrade Wizard
  5. Recover from Failed Steps
  6. Remove CDH 5 Packages
  7. Finalize HDFS Rolling Upgrade
  8. Upgrade Wizard Actions

Before You Begin

  • Ensure Java 1.7 is installed across the cluster. For installation instructions and recommendations, see Upgrading to Oracle JDK 1.7 in a Cloudera Manager Deployment, and make sure you have read Known Issues and Workarounds in Cloudera Manager 5 before you proceed with the upgrade.
  • To avoid lots of alerts during the upgrade process, you can enable maintenance mode on your cluster before you start the upgrade. This will stop email alerts and SNMP traps from being sent, but will not stop checks and configuration validations from being made. Be sure to exit maintenance mode when you have finished the upgrade to re-enable Cloudera Manager alerts.
  • Whenever upgrading Impala, whether in CDH or a standalone parcel or package, check your SQL against the newest reserved words listed in incompatible changes. If upgrading across multiple versions or in case of any problems, check against the full list of Impala keywords.

Ensure High Availability is Enabled

To enable high availability, see HDFS High Availability for instructions. You do not need to enable automatic failover for rolling restart to work, though you can enable it if you want. Automatic failover does not affect the rolling restart operation. If you have JobTracker high availability configured, Cloudera Manager will fail over the JobTracker during the rolling restart, but this is not a requirement for performing a rolling upgrade.

Back up HDFS Metadata

If upgrading from:
  • CDH 5.0 or 5.1 to 5.2 or higher
  • CDH 5.2 or 5.3 to 5.4 or higher

Back up HDFS metadata using the following command:

hdfs dfsadmin -fetchImage local directory

Run the Upgrade Wizard

  1. Log into the Cloudera Manager Admin console.
  2. From the Home > Status tab, click next to the cluster name and select Upgrade Cluster. The Upgrade Wizard starts.
  3. If the option to pick between packages and parcels displays, select the Use Parcels option.
  4. In the Choose CDH Version (Parcels) field, select the CDH version. If there are no qualifying parcels, click the Modify the Remote Parcel Repository URLs link to go to the Parcel Configuration Settings page where you can add the locations of parcel repositories. Click Continue.
  5. Read the notices for steps you must complete before upgrading, click the Yes, I ... checkboxes after completing the steps, and click Continue.
  6. Cloudera Manager checks that hosts have the correct software installed. Click Continue.
  7. The selected parcels are downloaded and distributed. Click Continue.
  8. The Host Inspector runs and displays the CDH version on the hosts. Click Continue. The Choose Upgrade Procedure displays.
  9. Choose the type of upgrade and restart:
    • Rolling Restart (Default) - Cloudera Manager upgrades services and performs a rolling restart.
      1. Choose the batch size (default 1) described in Performing a Cluster-Level Rolling Restart and click Advanced Options to specify additional rolling restart options. Services that do not support rolling restart undergo a normal restart, and are not available during the restart process.
      2. Click Continue. The Upgrade Cluster Command screen displays the result of the commands run by the wizard as it shuts down services, activates the new parcel, upgrades services as necessary, deploys client configuration files, restarts services, and performs a rolling restart of the services that support it.
    • Full Cluster Restart - Cloudera Manager performs all service upgrades and restarts the cluster, during which services are not available.
      1. Click Continue. The Upgrade Cluster Command screen displays the result of the commands run by the wizard as it shuts down all services, activates the new parcel, upgrades services as necessary, deploys client configuration files, and restarts services.
      2. Click Continue. The wizard reports the result of the upgrade.
    • Manual Upgrade - Cloudera Manager configures the cluster to the specified CDH version but performs no upgrades or service restarts. Manually doing the upgrade is difficult and is for advanced users only.
      1. Click Continue. Cloudera Manager displays links to documentation describing the required upgrade steps.
  10. Click Continue.
  11. Click Finish to return to the Home page.

Recover from Failed Steps

The actions performed by the upgrade wizard are listed in Upgrade Wizard Actions. If any of the steps in the Upgrade Cluster Command screen fails, complete the steps as described in that section before proceeding.

Remove CDH 5 Packages

If your previous installation of CDH was done using packages, remove those packages on all hosts on which you installed the parcels and refresh the symlinks so that clients will run the new software versions. Skip this step if your previous installation was using parcels.

  1. If your Hue service uses the embedded SQLite DB, back up /var/lib/hue/desktop.db to a location that is not /var/lib/hue as this directory is removed when the packages are removed.
  2. Uninstall the CDH packages on each host:
    • Not including Impala and Search
      Operating System Command
      RHEL 
      $ sudo yum remove bigtop-utils bigtop-jsvc bigtop-tomcat hue-common sqoop2-client
      SLES 
      $ sudo zypper remove bigtop-utils bigtop-jsvc bigtop-tomcat hue-common sqoop2-client
      Ubuntu or Debian 
      $ sudo apt-get purge bigtop-utils bigtop-jsvc bigtop-tomcat hue-common sqoop2-client
    • Including Impala and Search
      Operating System Command
      RHEL 
      $ sudo yum remove 'bigtop-*' hue-common impala-shell solr-server sqoop2-client hbase-solr-doc avro-libs crunch-doc avro-doc solr-doc
      SLES 
      $ sudo zypper remove 'bigtop-*' hue-common impala-shell solr-server sqoop2-client hbase-solr-doc avro-libs crunch-doc avro-doc solr-doc
      Ubuntu or Debian 
      $ sudo apt-get purge 'bigtop-*' hue-common impala-shell solr-server sqoop2-client hbase-solr-doc avro-libs crunch-doc avro-doc solr-doc
  3. Restart all the Cloudera Manager Agents to force an update of the symlinks to point to the newly installed components on each host: 
    $ sudo service cloudera-scm-agent restart
  4. If your Hue service uses the embedded SQLite DB, restore the DB you backed up:
    1. Stop the Hue service.
    2. Copy the backup from the temporary location to the newly created Hue database directory, /var/lib/hue.
    3. Start the Hue service.

Finalize HDFS Rolling Upgrade

If upgrading from:
  • CDH 5.0 or 5.1 to 5.2 or higher
  • CDH 5.2 or 5.3 to 5.4 or higher
Finalize the HDFS metadata upgrade. To determine when finalization is warranted, run important workloads and ensure they are successful. Once you have finalized the upgrade, it is not possible to roll back to a previous version of HDFS without using backups. Verifying that you are ready to finalize the upgrade can take a long time. Make sure you have enough free disk space, keeping in mind the following:
  • Deleting files does not free up disk space.
  • Using the balancer causes all moved replicas to be duplicated.
  • All on-disk data representing the NameNodes metadata is retained, which could more than double the amount of space required on the NameNode and JournalNode disks.
To finalize the metadata upgrade:
  1. Go to the HDFS service.
  2. Select Actions > Finalize Rolling Upgrade and click Finalize Rolling Upgrade to confirm.

Upgrade Wizard Actions

Do the steps in this section only if the upgrade wizard reports a failure.

Back up Metastore Databases

Back up the Hive, Sentry, and Sqoop metastore databases.
  1. For each affected service:
    1. If not already stopped, stop the service.
    2. Back up the database. See Backing Up Databases.

Upgrade HDFS Metadata

If upgrading from:
  • CDH 5.0 or 5.1 to 5.2 or higher
  • CDH 5.2 or 5.3 to 5.4 or higher
  1. Start the ZooKeeper service.
  2. Go to the HDFS service.
  3. Select Actions > Upgrade HDFS Metadata and click Upgrade HDFS Metadata to confirm.

Upgrade the Hive Metastore Database

Required for the following upgrades:
  • CDH 5.0 or 5.1 to 5.2 or higher
  • CDH 5.3 to 5.4 or higher
  1. Go to the Hive service.
  2. Select Actions > Stop and click Stop to confirm.
  3. Select Actions > Upgrade Hive Metastore Database Schema and click Upgrade Hive Metastore Database Schema to confirm.
  4. If you have multiple instances of Hive, perform the upgrade on each metastore database.

Upgrade the Oozie ShareLib

  1. Go to the Oozie service.
  2. Select Actions > Start and click Start to confirm.
  3. Select Actions > Install Oozie ShareLib and click Install Oozie ShareLib to confirm.

Upgrade Sqoop

  1. Go to the Sqoop service.
  2. Select Actions > Stop and click Stop to confirm.
  3. Select Actions > Upgrade Sqoop and click Upgrade Sqoop to confirm.

Upgrade the Sentry Database

Required for the following upgrades:
  • CDH 5.1 to 5.2 or higher
  • CDH 5.2 to 5.3 or higher
  • CDH 5.4 to 5.5 or higher
  1. Go to the Sentry service.
  2. Select Actions > Stop and click Stop to confirm.
  3. Select Actions > Upgrade Sentry Database Tables and click Upgrade Sentry Database Tables to confirm.

Upgrade Spark

  1. Go to the Spark service.
  2. Select Actions > Stop and click Stop to confirm.
  3. Select Actions > Install Spark JAR and click Install Spark JAR to confirm.
  4. Select Actions > Create Spark History Log Dir and click Create Spark History Log Dir to confirm.

Start Cluster Services

  1. On the Home > Status tab, click to the right of the cluster name and select Start.
  2. Click Start that appears in the next screen to confirm. The Command Details window shows the progress of starting services.

    When All services successfully started appears, the task is complete and you can close the Command Details window.

Deploy Client Configuration Files

  1. On the Home page, click to the right of the cluster name and select Deploy Client Configuration.
  2. Click the Deploy Client Configuration button in the confirmation pop-up that appears.
Page generated July 8, 2016.