This is the documentation for Cloudera Enterprise 5.8.x. Documentation for other versions is available at Cloudera Documentation.

DSSD D5 and Short-Circuit Reads

"Short-circuit" reads bypass the DataNode, allowing a client to read data from the disk directly, as long as the client is co-located with the data. Short-circuit reads provide a substantial performance boost to many applications and help improve HBase random read profile and Impala performance.

  Important:

Enabling short-circuit reads for HBase or Impala on an HDFS cluster that uses DSSD D5 DataNodes requires that the processes associated with these applications be granted hdfs group membership. When short-circuit reads are enabled for Impala (for example), Impala process that act as short-circuit read clients (like impalad) are able to read and write all data stored in the DSSD D5. Cloudera Manager applies the hdfs group membership on a per-service basis, and applications that do not require short-circuit reads or for which short-circuit reads have not been enabled will have the same granularity of access control as present on a traditional HDFS cluster. Whether short-circuit reads are enabled or not, access control that is enforced by the application rather than at the file system level is identical for DSSD D5 DataNode HDFS clusters and traditional HDFS clusters.

Short-circuit reads improve the performance of applications when enabled, but it is not required and can be disabled if the coarser file system access control permissions it implies are problematic.

Continue reading:

Enabling Short Circuit Reads for HBase and Impala

To enable short-circuit reads for use with DSSD D5 DataNodes, you must have installed the DSSD_SCR parcel during installation. If you have not yet installed this parcel, download, distribute, and activate the parcel. See Parcels. Short-circuit reads are enabled for HBase and Impala by default.

To enable short circuit reads for HBase:
  1. In the Cloudera Manager Admin Console, select Clusters > HBase > Configuration.
  2. Type “short” in the Search box.

    A set of short-circuit read parameters for HBase display.

  3. Select the Enable DSSD Short-Circuit Read property.
  4. You can leave the default values for the Libflood Short-Circuit Read CPU ID , Libflood Short-Circuit Read Command Queues, and Libflood Short-Circuit Read Command Queue Depth parameters or adjust them to fine tune HBase short-circuit reads for best performance.
  5. Click Save Changes to commit the changes.

    The Admin console indicates that there is a stale configuration.

  6. Restart the stale services as indicated. See Stale Configurations.
To enable short-circuit reads for Impala:
  1. In the Cloudera Manager Admin Console, select Clusters > Impala > Configuration.
  2. Type “short” in the Search box.

    A set of short-circuit read parameters for Impala display.

  3. Select the Enable DSSD Short-Circuit Read property.
  4. You can leave the default values for the Libflood Short-Circuit Read CPU ID , Libflood Short-Circuit Read Command Queues, and Libflood Short-Circuit Read Command Queue Depth parameters or adjust them to fine tune Impala short-circuit reads for best performance.
  5. Click Save Changes to commit the changes.

    The Admin console now indicates that there is a stale configuration.

  6. Restart the stale services as indicated. See Stale Configurations.

Disabling Short Circuit Reads for HBase and Impala

Short-circuit reads are enabled for HBase and Impala by default. To disable short-circuit reads for use with DSSD D5 DataNodes:

To disable short circuit reads for HBase:
  1. In the Cloudera Manager Admin Console, select Clusters > HBase > Configuration.
  2. Type “short” in the Search box.

    A set of short-circuit read parameters for HBase display.

  3. Clear the Enable DSSD Short-Circuit Read property.
  4. Click Save Changes to commit the changes.

    The Admin console indicates that there is a stale configuration.

  5. Restart the stale services as indicated. See Stale Configurations.
To disable short-circuit reads for Impala:
  1. In the Cloudera Manager Admin Console, select Clusters > Impala > Configuration.
  2. Type “short” in the Search box.

    A set of short-circuit read parameters for Impala display.

  3. Clear the Enable DSSD Short-Circuit Read property.
  4. Click Save Changes to commit the changes.

    The Admin console now indicates that there is a stale configuration.

  5. Restart the stale services as indicated. See Stale Configurations.
Page generated July 8, 2016.