Cloudera Navigator Key Trustee Server High Availability
Key Trustee Server high availability applies to read operations only. If either Key Trustee Server fails, the KeyProvider automatically retries fetching keys from the functioning server. New write operations (for example, creating new encryption keys) are not allowed unless both Key Trustee Servers are operational.
If a Key Trustee Server fails, the following operations are impacted:
- HDFS Encryption
- You cannot create new encryption keys for encryption zones.
- You can write to and read from existing encryption zones, but you cannot create new zones.
- Cloudera Navigator Encrypt
- You cannot register new Cloudera Navigator Encrypt clients.
- You can continue reading and writing encrypted data, including creating new mount points, using existing clients.
Cloudera recommends monitoring both Key Trustee Servers. If a Key Trustee Server fails catastrophically, restore it from backup to a new host with the same hostname and IP address as the failed host. See Backing Up and Restoring Key Trustee Server and Clients for more information. Cloudera does not support PostgreSQL promotion to convert a passive Key Trustee Server to an active Key Trustee Server.
Continue reading:
Configuring Key Trustee Server High Availability Using Cloudera Manager
For new installations, use the Set up HDFS Data At Rest Encryption wizard and follow the instructions in Enabling HDFS Encryption Using the Wizard. When prompted, make sure that the Enable High Availability option is selected.
- Stop the Key Trustee Server service ( ).
- Run the Set Up Key Trustee Server Database command ( ).
- Run the following command on the Active Key Trustee Server:
$ sudo rsync -zav --exclude .ssl /var/lib/keytrustee/.keytrustee root@keytrustee02.example.com:/var/lib/keytrustee/.
Replace keytrustee02.example.com with the hostname of the Passive Key Trustee Server.
- Run the following command on the Passive Key Trustee Server:
$ sudo ktadmin init
- Start the Key Trustee Server service (Important: Starting or restarting the Key Trustee Server service attempts to start the Active Database and Passive Database roles. If the Active Database is not running when the Passive Database attempts to start, the Passive Database fails to start. If this occurs, manually restart the Passive Database role after confirming that the Active Database role is running.
).
- Enable synchronous replication ( ).
- Restart the Key Trustee Server service ( ).
For parcel-based Key Trustee Server releases 5.8 and higher, Cloudera Manager automatically backs up Key Trustee Server (using the ktbackup.sh script) after adding the Key Trustee Server service. It also schedules automatic backups using cron. For package-based installations, you must manually back up Key Trustee Server and configure a cron job.
Cloudera Manager configures cron to run the backup script hourly. The latest ten backups are retained in /var/lib/keytrustee in cleartext. For information about using the backup script and configuring the cron job (including how to encrypt backups), see Backing Up Key Trustee Server and Key Trustee KMS Using the ktbackup.sh Script.
Configuring Key Trustee Server High Availability Using the Command Line
Install and configure a second Key Trustee Server following the instructions in Installing Cloudera Navigator Key Trustee Server.
$ sudo -u keytrustee ssh-keygen -t rsa /var/lib/keytrustee/.ssh/id_rsa
$ sudo ktadmin init --external-address keytrustee01.example.com $ sudo rsync -zav --exclude .ssl /var/lib/keytrustee/.keytrustee root@keytrustee02.example.com:/var/lib/keytrustee/. $ sudo ktadmin db --bootstrap --port 11381 --pg-rootdir /var/lib/keytrustee/db --slave keytrustee02.example.com ## For RHEL/CentOS 7, use 'sudo systemctl [stop|start] <service_name>' instead of 'sudo service <service_name> [stop|start]' ## $ sudo service keytrustee-db stop $ sudo service keytrustee-db start $ sudo service keytrusteed start
Replace keytrustee01.example.com with the fully qualified domain name (FQDN) of the active Key Trustee Server. Replace keytrustee02.example.com with the FQDN of the passive Key Trustee Server. Replace /var/lib/keytrustee/db with the path to the directory you want to use to store the PostgreSQL database.
To use a different port for the database, modify the ktadmin init and ktadmin db commands as follows:
$ sudo ktadmin init --external-address keytrustee01.example.com --db-connect postgresql://localhost:<port>/keytrustee?host=/tmp $ sudo ktadmin db --bootstrap --port <port> --pg-rootdir /var/lib/keytrustee/db --slave keytrustee02.example.com
ARGS="--pg-rootdir /path/to/db"
The ktadmin init command generates a self-signed certificate that the Key Trustee Server uses for HTTPS communication.
$ sudo ktadmin init-slave --master keytrustee01.example.com --pg-rootdir /var/lib/keytrustee/db --no-import-key --no-start ## For RHEL/CentOS 7, use 'sudo systemctl [stop|start] <service_name>' instead of 'sudo service <service_name> [stop|start]' ## $ sudo service keytrustee-db start $ sudo ktadmin init --external-address keytrustee02.example.com $ sudo service keytrusteed start
Replace keytrustee02.example.com with the fully qualified domain name (FQDN) of the passive Key Trustee Server. Replace keytrustee01.example.com with the FQDN of the active Key Trustee Server. Replace /var/lib/keytrustee/db with the path to the directory you want to use to store the PostgreSQL database.
To use a different port for the database, modify the ktadmin init-slave command as follows:
$ sudo ktadmin init-slave --master keytrustee01.example.com --pg-port <port> --pg-rootdir /var/lib/keytrustee/db --no-import-key --no-start
ARGS="--pg-rootdir /path/to/db"
The ktadmin init-slave command performs an initial database sync by running the pg_basebackup command. The database directory must be empty for this step to work. For information on performing an incremental backup, see the PostgreSQL documentation.
The ktadmin init command generates a self-signed certificate that the Key Trustee Server uses for HTTPS communication. Instructions for using alternate certificates (for example, if you have obtained certificates from a trusted Certificate Authority) are provided later.
Enable Synchronous Replication
Key Trustee Server high availability requires synchronous replication to ensure that all rows in the database are inserted in at least two hosts, which protects against key loss.
$ sudo ktadmin enable-synchronous-replication --pg-rootdir /var/lib/keytrustee/db
If you modified the default database location, replace /var/lib/keytrustee/db with the modified path.
(Optional) Replace Self-Signed Certificates with CA-Signed Certificates
If you have a CA-signed certificate for Key Trustee Server, see Managing Key Trustee Server Certificates for instructions on how to replace the self-signed certificates.
Recovering a Key Trustee Server
If a Key Trustee Server fails, restore it from backup as soon as possible. If the Key Trustee Server hosts fails completely, make sure that you restore the Key Trustee Server to a new host with the same hostname and IP address as the failed host.
For more information, see Backing Up and Restoring Key Trustee Server and Clients.
<< MapReduce (MRv1) JobTracker High Availability | ©2016 Cloudera, Inc. All rights reserved | Key Trustee KMS High Availability >> |
Terms and Conditions Privacy Policy |