1. Run the upgrade command in -check mode, and examine the output.

   $ /usr/lib/hbase/bin/hbase upgrade -check

   Your output should be similar to the following:

   Tables Processed:
   hdfs://localhost:41020/myHBase/.META.
   hdfs://localhost:41020/myHBase/usertable
   hdfs://localhost:41020/myHBase/TestTable
   hdfs://localhost:41020/myHBase/t
   
   Count of HFileV1: 2
   HFileV1:
   hdfs://localhost:41020/myHBase/usertable /fa02dac1f38d03577bd0f7e666f12812/family/249450144068442524
   hdfs://localhost:41020/myHBase/usertable /ecdd3eaee2d2fcf8184ac025555bb2af/family/249450144068442512
   
   Count of corrupted files: 1
   Corrupted Files:
   hdfs://localhost:41020/myHBase/usertable/fa02dac1f38d03577bd0f7e666f12812/family/1
   Count of Regions with HFileV1: 2
   Regions to Major Compact:
   hdfs://localhost:41020/myHBase/usertable/fa02dac1f38d03577bd0f7e666f12812
   hdfs://localhost:41020/myHBase/usertable/ecdd3eaee2d2fcf8184ac025555bb2af

   In the example above, you can see that the script has detected two HFile v1 files, one corrupt file and the regions to major compact.

   By default, the script scans the root directory, as defined by hbase.rootdir. To scan a specific directory, use the --dir option. For example, the following command scans the /myHBase/testTable directory.

   /usr/lib/hbase/bin/hbase upgrade --check --dir /myHBase/testTable

2. Trigger a major compaction on each of the reported regions. This major compaction rewrites the files from HFile v1 to HFile v2 format. To run the major compaction, start HBase Shell and issue the major_compact command.

   $ /usr/lib/hbase/bin/hbase shell
   hbase> major_compact 'usertable'

   You can also do this in a single step by using the echo shell built-in command.

   $ echo "major_compact 'usertable'" | /usr/lib/hbase/bin/hbase shell

3. Once all the HFileV1 files have been rewritten, running the upgrade script with the -check option again will return a "No HFile v1 found" message. It is then safe to proceed with the upgrade.

Shut down your CDH 4 HBase cluster before you run the upgrade script in -execute mode.

To shut down HBase gracefully:

1. Stop the REST and Thrift server and clients, then stop the cluster.
   1. Stop the Thrift server and clients:

      sudo service hbase-thrift stop

      Stop the REST server:

      sudo service hbase-rest stop

   2. Stop the cluster by shutting down the master and the RegionServers:
      1. Use the following command on the master node:

         sudo service hbase-master stop

      2. Use the following command on each node hosting a RegionServer:

         sudo service hbase-regionserver stop

2. Stop the ZooKeeper Server:

   $ sudo service zookeeper-server stop

1. To remove HBase on Red-Hat-compatible systems:

   $ sudo yum remove hadoop-hbase

   To remove HBase on SLES systems:

   $ sudo zypper remove hadoop-hbase

   To remove HBase on Ubuntu and Debian systems:

   $ sudo apt-get purge hadoop-hbase

     Warning:

   If you are upgrading an Ubuntu or Debian system from CDH3u3 or lower, you must use apt-get purge (rather than apt-get remove) to make sure the re-install succeeds, but be aware that apt-get purge removes all your configuration data. If you have modified any configuration files, DO NOT PROCEED before backing them up.

   CAUTION:
   On Ubuntu systems, make sure you remove HBase before removing ZooKeeper; otherwise your HBase configuration will be deleted. This is because hadoop-hbase depends on hadoop-zookeeper, and so purging hadoop-zookeeper will purge hadoop-hbase.
2. Follow the instructions for installing the new version of HBase at HBase Installation.

  Important: Before you proceed with Step 4, upgrade your CDH 4 cluster to CDH 5. See Upgrading to CDH 5 for instructions.

This step executes the actual upgrade process. It has a verification step which checks whether or not the Master, RegionServer and backup Master znodes have expired. If not, the upgrade is aborted. This ensures no upgrade occurs while an HBase process is still running. If your upgrade is aborted even after shutting down the HBase cluster, retry after some time to let the znodes expire. Default znode expiry time is 300 seconds.

As mentioned earlier, ZooKeeper and HDFS should be available. If ZooKeeper is managed by HBase, then use the following command to start ZooKeeper.

/usr/lib/hbase/bin/hbase-daemon.sh start zookeeper

The upgrade involves three steps:

• Upgrade Namespace: This step upgrades the directory layout of HBase files.
• Upgrade Znodes: This step upgrades /hbase/replication (znodes corresponding to peers, log queues and so on) and table znodes (keep table enable/disable information). It deletes other znodes.
• Log Splitting: In case the shutdown was not clean, there might be some Write Ahead Logs (WALs) to split. This step does the log splitting of such WAL files. It is executed in a “non distributed mode”, which could make the upgrade process longer in case there are too many logs to split. To expedite the upgrade, ensure you have completed a clean shutdown.

$ /usr/lib/hbase/bin/hbase upgrade -execute

Your output should be similar to the following:

Starting Namespace upgrade
Created version file at hdfs://localhost:41020/myHBase with version=7
Migrating table testTable to hdfs://localhost:41020/myHBase/.data/default/testTable
…..
Created version file at hdfs://localhost:41020/myHBase with version=8
Successfully completed NameSpace upgrade.
Starting Znode upgrade
….
Successfully completed Znode upgrade
Starting Log splitting
…
Successfully completed Log splitting

The output of the -execute command can either return a success message as in the example above, or, in case of a clean shutdown where no log splitting is required, the command would return a "No log directories to split, returning" message. Either of those messages indicates your upgrade was successful.

  Important: Configuration files

• If you install a newer version of a package that is already on the system, configuration files that you have modified will remain intact.
• If you uninstall a package, the package manager renames any configuration files you have modified from <file> to <file>.rpmsave. If you then re-install the package (probably to install a new version) the package manager creates a new <file> with applicable defaults. You are responsible for applying any changes captured in the original configuration file to the new configuration file. In the case of Ubuntu and Debian upgrades, you will be prompted if you have made changes to a file for which there is a new version. For details, see Automatic handling of configuration files by dpkg.

CDH 5 introduces namespaces for HBase tables. As a result of the upgrade, all tables are automatically assigned to namespaces. The root, meta, and acl tables are added to the hbase system namespace. All other tables are assigned to the default namespace.

To move a table to a different namespace, take a snapshot of the table and clone it to the new namespace. After the upgrade, do the snapshot and clone operations before turning the modified application back on.

  Warning: Do not move datafiles manually, as this can cause data corruption that requires manual intervention to fix.

Recompile any coprocessors and custom JARs, so that they will work with the new version of HBase.