Upgrading Cloudera Navigator Key Trustee Server 3.8 to 5.5 Using the ktupgrade Script
Cloudera provides a Python script (ktupgrade) to simplify upgrading Key Trustee Server 3.8 to 5.5. The script upgrades package-based Key Trustee Server 3.8 to package-based Key Trustee Server 5.5 and switches the web server from Apache to CherryPy. After the upgrade completes, you must manually migrate Key Trustee Server to use parcels and be managed by Cloudera Manager.
To upgrade from 3.x to 5.5 manually, you must first upgrade to 5.4, and then upgrade to 5.5:
- Upgrading Cloudera Navigator Key Trustee Server 3.x to 5.4.x
- Upgrading Cloudera Navigator Key Trustee Server 5.4.x or 5.5.x to the Latest Release
Continue reading:
Prerequisites
- Before upgrading Key Trustee Server, upgrade Cloudera Manager and CDH. See Upgrading Cloudera Manager and Upgrading CDH and Managed Services Using Cloudera Manager. If you are upgrading Key Trustee Server to a version higher than 5.5.2, you can upgrade Cloudera Manager and CDH directly to the version you want before continuing; you do not need to upgrade Cloudera Manager and CDH to 5.5 and complete the Key Trustee Server upgrade before upgrading Cloudera Manager and CDH to a higher version. The Cloudera Manager version must be equal to or higher than the Key Trustee Server version. See Product Compatibility Matrix for Cloudera Navigator Encryption for more information.
- If you are using HDFS Transparent Encryption with Key Trustee Server, upgrade Key Trustee KMS. See Upgrading Key Trustee KMS for instructions.
- You must run the ktupgrade script as root.
- The ktupgrade script uses yum to upgrade Key Trustee Server. If the Key Trustee Server host does not have Internet access,
you must download the Key Trustee Server dependencies from a host with Internet access and copy them to the Key Trustee Server host:
- Create a temporary directory to store the packages:
$ mkdir tmp-keytrustee
- Download the bigtop-utils package from the CDH repository:
$ sudo wget -P tmp-keytrustee <url>
Replace <url> with the URL corresponding to the Key Trustee Server version to which you are upgrading:
Table 1. URL for bigtop-utils Package Key Trustee Server Version URL 5.5.2 http://archive.cloudera.com/cdh5/redhat/6/x86_64/cdh/5.5.2/RPMS/noarch/bigtop-utils-0.7.0+cdh5.5.2+0-1.cdh5.5.2.p0.10.el6.noarch.rpm 5.5.0 http://archive.cloudera.com/cdh5/redhat/6/x86_64/cdh/5.5.0/RPMS/noarch/bigtop-utils-0.7.0+cdh5.5.0+0-1.cdh5.5.0.p0.15.el6.noarch.rpm - Download the python-paste and python-cherrypy packages:
$ sudo yum install yum-downloadonly $ sudo yum install --downloadonly --downloaddir=tmp-keytrustee/ python-paste python-cherrypy
- Copy the packages to the Key Trustee Server host:
$ sudo scp tmp-keytrustee/*.rpm <username>@kts01.example.com:/path/to/tmp-keytrustee
Replace kts01.example.com with the hostname of the active Key Trustee Server, and /path/to/tmp-keytrustee with the path to a directory to which you have access.
- Create a temporary directory to store the packages:
Download the ktupgrade Script and Repository Tarball
- Download the ktupgrade script on the active Key Trustee Server host:
$ sudo wget http://archive.gazzang.com/keytrustee/ktupgrade
If the Key Trustee Server host does not have Internet access, run the command on an Internet-connected host, and then copy the file to the active Key Trustee Server host.
- Download the repository tarball for Key Trustee Server 5.5.0 or
5.5.2:
- Select Packages from the SELECT DOWNLOAD TYPE drop-down menu.
- Select your operating system from the SELECT AN OS drop-down menu.
- Click DOWNLOAD NOW.
- Copy the downloaded file to the active Key Trustee Server host. Make sure you put the repository tarball and ktupgrade script in the same directory.
Run the ktupgrade Script
If your environment does not allow root to log in over SSH, contact Cloudera Support for assistance.
Upgrade the Active Key Trustee Server Using the ktupgrade Script
- On the active Key Trustee Server host, change to the directory that contains the ktupgrade script and the repository tarball:
# cd /path/to/tmp-keytrustee
If the host does not have Internet access, make sure that the dependency files you downloaded in Prerequisites are in the same directory as the script and tarball.
- Make sure the script is executable:
# chmod a+x ktupgrade
- Run the ktupgrade script as follows:
# ./ktupgrade upgrade-active-kts key-trustee-server-5.5.2-el6.tar.gz
Replace key-trustee-server-5.5.2-el6.tar.gz with the file name of the repository tarball you downloaded in Download the ktupgrade Script and Repository Tarball.
Downgrade Key Trustee Server Using the ktupgrade Script
If you experience any problems upgrading Key Trustee Server, you can use the script to downgrade to your previous version. Run the following command on the active Key Trustee Server:
# cd /path/to/tmp-keytrustee # ./ktupgrade downgrade-active-kts
Migrate Key Trustee Server to Cloudera Manager
Minimum Required Role: Cluster Administrator (also provided by Full Administrator)
Before continuing, you must create an internal repository for the Key Trustee Server parcel. For instructions on creating internal repositories (including Cloudera Manager, CDH, and Cloudera Navigator encryption components), see Creating and Using a Remote Parcel Repository for Cloudera Manager.
After creating the internal Key Trustee Server parcel repository, do the following:
- Create a new cluster in Cloudera Manager containing only the Key Trustee Server hosts. This enables multiple clusters to share the same Key Trustee Server and avoids restarting Key Trustee Server when restarting a cluster. See Adding and Deleting Clusters for instructions on how to create a new cluster in Cloudera Manager.
- Download, distribute, and activate the Key Trustee Server parcel, following the instructions in Managing Parcels. After you activate the Key Trustee Server parcel, Cloudera Manager prompts you to restart the cluster. Click the Close button to ignore this prompt. You do not need to restart the cluster after installing Key Trustee Server.
- Stop the active and passive Key Trustee Server web servers by running the following command on all Key Trustee Server hosts:
$ sudo -u keytrustee service keytrusteed stop
- Stop the active Key Trustee Server database by running the following command on the active Key Trustee Server:
$ sudo -u keytrustee service keytrustee-db stop
Warning: Do not stop the passive Key Trustee Server database. If it is stopped, start it before proceeding by running the following command on the passive Key Trustee Server:$ sudo -u keytrustee service keytrustee-db start
- Add the Key Trustee Server service to your cluster, following the instructions in Adding a Service. When customizing role assignments, assign the Active Key Trustee Server and Active Database roles to the active Key Trustee Server host, and the Passive Key Trustee Server and Passive Database roles to the passive Key Trustee Server host.
- Stop the passive Key Trustee Server database. Run the following command on the passive Key Trustee Server:
$ sudo -u keytrustee service keytrustee-db stop
- Restart the Key Trustee Server service (Important: Starting or restarting the Key Trustee Server service attempts to start the Active Database and Passive Database roles. If the Active Database is not running when the Passive Database attempts to start, the Passive Database fails to start. If this occurs, manually restart the Passive Database role after confirming that the Active Database role is running.
).
- (High Availability Key Trustee Servers Only) Enable synchronous replication. Run the following command on the active Key Trustee Server:
$ sudo -u keytrustee ktadmin enable-synchronous-replication --pg-rootdir /var/lib/keytrustee/db
Validate Key Operations
$ curl -k https://keytrustee.example.com:11371/?a=fingerprint 4096R/4EDC46882386C827E20DEEA2D850ACA33BEDB0D1
Replace keytrustee.example.com with the fully qualified domain name (FQDN) of each Key Trustee Server you are validating.
If you are using Key Trustee Server as the backing key store for HDFS Transparent Encryption, run the following commands to verify that Hadoop key operations are successful:
$ hadoop key create hadoop_test_key $ hadoop key list $ hadoop key delete hadoop_test_key
Updating Key Trustee Server Clients
After upgrading Key Trustee Server to 5.4 or higher, you must configure Key Trustee Server clients (namely Key Trustee KMS and Cloudera Navigator Encrypt) to communicate with Key Trustee Server over the new ports:
- Key Trustee KMS
Add the following entries to the Key Trustee KMS advanced configuration snippet (
):<property> <name>cloudera.trustee.keyprovider.hkpport</name> <value>hkp_port_number</value> <description> Indicates the HTTP port on which Key Trustee Server clients should request public keys. On Key Trustee Server 3.8 (Apache webserver-based) servers, this is usually port 80 (unencrypted). On Key Trustee Server 5.4 and higher (CherryPy-based) servers, this is usually port 11371 (SSL-encrypted). </description> </property> <property> <name>cloudera.trustee.keyprovider.ktsport</name> <value>kts_port_number</value> <description> Indicates the HTTPS port on which the client sends and receives Key Trustee Server protocol messages. On Key Trustee Server 3.8 (Apache webserver-based) servers, this is usually port 443 (SSL-encrypted). On Key Trustee Server 5.4 and higher (CherryPy-based) servers, this is usually port 11371 (SSL-encrypted). </description> </property> <property> <name>cloudera.trustee.keyprovider.hkpssl</name> <value>boolean</value> <description> Indicates whether the client should communicate with the HKP server over an SSL-encrypted (true) or unencrypted (false) channel. On Key Trustee Server 3.8 (Apache webserver-based) servers, this is usually false (unencrypted). On Key Trustee Server 5.4 and higher (CherryPy-based) servers, this is usually true (SSL-encrypted). </description> </property>
- Cloudera Navigator Encrypt
See Updating Key Trustee Server Ports for instructions on updating Cloudera Navigator Encrypt to use the new ports.
(Optional) Upgrade to a Higher Release
If you are upgrading Key Trustee Server to a version higher than 5.5.2, continue to Upgrading Cloudera Navigator Key Trustee Server 5.4.x or 5.5.x to the Latest Release.