This is the documentation for Cloudera Enterprise 5.8.x. Documentation for other versions is available at Cloudera Documentation.

Enabling and Disabling Fair Scheduler Preemption

You can enable the Fair Scheduler to preempt applications in other pools if a pool's fair or minimum share is not met for some period of time. When you create a pool, you can specify whether preemption is allowed and whether a specific pool can be preempted. Fair scheduler preemption is controlled by several properties. For more information, see Configuring the Fair Scheduler.

Enabling and Disabling Preemption Using Cloudera Manager

Minimum Required Role: Configurator (also provided by Cluster Administrator, Full Administrator)

Enabling Preemption

  1. Select Clusters > Cluster name > Dynamic Resource Pool Configuration. If the cluster has a YARN service, the YARN > Resource Pools tab displays. If the cluster has an Impala service enabled for dynamic resource pools, the Impala Admission Control > Resource Pools tab displays.
  2. Click Default Settings.
  3. Click the Enable Fair Scheduler Preemption link.
  4. Select the ResourceManager Default Group checkbox.
  5. Click Save Changes to commit the changes.
  6. Return to the Home page by clicking the Cloudera Manager logo.
  7. Click to invoke the cluster restart wizard.
  8. Click Restart Stale Services.
  9. Click Restart Now.
  10. Click Finish.

Disabling Preemption

  1. Select Clusters > Cluster name > Dynamic Resource Pool Configuration. If the cluster has a YARN service, the YARN > Resource Pools tab displays. If the cluster has an Impala service enabled for dynamic resource pools, the Impala Admission Control > Resource Pools tab displays.
  2. Click the Other Settings button.
  3. Clear the Enable Fair Scheduler Preemption checkbox.
  4. Click Save Changes to commit the changes.
  5. Return to the Home page by clicking the Cloudera Manager logo.
  6. Click to invoke the cluster restart wizard.
  7. Click Restart Stale Services.
  8. Click Restart Now.
  9. Click Finish.

Enabling and Disabling Preemption for a Pool

  1. Select Clusters > Cluster name > Dynamic Resource Pool Configuration. If the cluster has a YARN service, the YARN > Resource Pools tab displays. If the cluster has an Impala service enabled for dynamic resource pools, the Impala Admission Control > Resource Pools tab displays.
  2. In a pool row, click Edit.
  3. Click the Preemption tab.
  4. Select or clear Allow Preemption From.
  5. Click Save.
  6. Click Refresh Dynamic Resource Pools.

Enabling and Disabling Preemption Using the Command Line

  Important:
  • If you use Cloudera Manager, do not use these command-line instructions.
  • This information applies specifically to CDH 5.8.x. If you use a lower version of CDH, see the documentation for that version located at Cloudera Documentation.
To enable and disable preemption, set yarn.scheduler.fair.preemption in yarn-site.xml: 
<property>
    <name>yarn.scheduler.fair.preemption</name>
    <value>true</value>
</property>
To enable or disable a specific pool from being preempted, set the allowPreemptionFrom property in fair-scheduler.xml. The following example disables preemption from the important pool:
<queue name="important">
    <weight>1.0</weight>
    <allowPreemptionFrom>false</allowPreemptionFrom>
    <schedulingPolicy>drf</schedulingPolicy>
    <aclSubmitApps>*</aclSubmitApps>
    <aclAdministerApps>*</aclAdministerApps>
</queue>
Page generated July 8, 2016.