List of Pages in Category HDFS (95 pages)
Hadoop Distributed File System (HDFS)
A user space filesystem designed for storing very large files with streaming data access patterns, running on clusters of industry-standard machines. HDFS defines three components:
- NameNode - Maintains the namespace tree for HDFS and a mapping of file blocks to DataNodes where the data is stored. A simple HDFS cluster can have only one primary NameNode, supported by a secondary NameNode that periodically compresses the NameNode edits log file that contains a list of HDFS metadata modifications. This reduces the amount of disk space consumed by the log file on the NameNode, which also reduces the restart time for the primary NameNode. A high availability cluster contains two NameNodes: active and standby.
- DataNode - Stores data in a Hadoop cluster and is the name of the daemon that manages the data. File data is replicated on multiple DataNodes for reliability and so that localized computation can be executed near the data.
- JournalNode - Maintains a directory to log the modifications to the namespace metadata when using the Quorum-based Storage mechanism for providing high availability. During failover, the NameNode standby ensures that it has applied all of the edits from the JournalNodes before promoting itself to the active state.
*
A
- About HttpFS
- Adding and Configuring an NFS Gateway
- Adding HttpFS
- Administering an HDFS High Availability Cluster
- Authorization With Apache Sentry
B
C
- Changing a Nameservice Name for Highly Available HDFS Using Cloudera Manager
- Configuring CDH Services for HDFS Encryption
- Configuring Centralized Cache Management in HDFS
- Configuring Encrypted HDFS Data Transport
- Configuring Encryption
- Configuring Encryption for Data Spills
- Configuring Hardware for HDFS HA
- Configuring HDFS Trash
- Configuring Heterogeneous Storage in HDFS
- Configuring HttpFS
- Configuring Mountable HDFS
- Configuring Other CDH Components to Use HDFS HA
- Configuring Proxy Users to Access HDFS
- Configuring Short-Circuit Reads
- Configuring Storage Balancing for DataNodes
- Configuring Storage Directories for DataNodes
- Configuring the Metastore to Use HDFS High Availability
- Configuring TLS/SSL for HDFS, YARN and MapReduce
- Configuring TLS/SSL for HttpFS
D
- DataNode Metrics
- Deploying Cloudera Search
- Deploying HDFS on a Cluster
- Detecting and Correcting HDFS Block Skew Conditions
- Directory Usage Report
- Disabling and Redeploying HDFS HA
- Disk Usage Reports
- Downloading HDFS Directory Access Permission Reports
- Downloading HDFS Directory Access Permission Reports
- DSSD D5 and Short-Circuit Reads
- DSSD DataNode Metrics
E
F
H
- HDFS Balancers
- HDFS Cache Directive Metrics
- HDFS Cache Pool Metrics
- HDFS Extended ACLs
- HDFS High Availability
- HDFS Metrics
- HDFS Replication
- HDFS Transparent Encryption
- HdfsFindTool
- Hedged Reads
- How Impala Fits Into the Hadoop Ecosystem
- HttpFS Authentication
- HttpFS Installation
- HttpFS Packaging
- HttpFS Prerequisites
I
- Installing CDH 5 with MRv1 on a Single Linux Host in Pseudo-distributed mode
- Installing CDH 5 with YARN on a Single Linux Host in Pseudo-distributed mode
- Installing HttpFS
- Introduction to HDFS High Availability
J
L
M
- Managing and Monitoring Federated HDFS
- Managing Encryption Keys and Zones
- Managing Federated Nameservices
- Managing HDFS
- Managing HDFS Snapshots
- MapReduce (MRv1) JobTracker High Availability
N
O
- Optimizing Performance for HDFS Transparent Encryption
- Optional Step 10: Configuring a secure HDFS NFS Gateway
- Optional Step 8: Configuring Security for HDFS High Availability
P
S
- Schema
- SecondaryNameNode Metrics
- Securing Impala Data and Log Files
- Setting HDFS Quotas
- Sizing NameNode Heap Memory
- Snapshots
- Starting the HttpFS Server
- Step 14: Create the HDFS Superuser Principal
- Step 14: Set the Sticky Bit on HDFS Directories
- Step 5: Create the HDFS Superuser
- Step 7: Configure Secure HDFS
- Stopping the HttpFS Server
- Synchronizing HDFS ACLs and Sentry Permissions
T
- Testing the Flume HDFS Sink Configuration
- The File Browser
- Troubleshooting HDFS Encryption
- Tuning the HDFS Block Size for DSSD Mode
U
© 2016 Cloudera, Inc. All rights reserved | ||
Terms and Conditions Privacy Policy |