Adjust Cgroup configuration to account for longer Cgroup deletion and make CPU utilization configurable

Description

https://www.gresearch.co.uk/2019/01/28/hadoop-yarn-cgroup-stability-issues/

Cgroup only gets deleted if a process assigned to the Cgroup has exited. If it has not exited after the timeout then the Cgroup will not get deleted. We should increase the timeout to reduce the risk of accumulating Cgroups. yarn.nodemanager.linux-container-executor.cgroups.delete-timeout-ms will be set to 5000ms.

Also the accumulated CPU utilization of NM containers should not always be set to 100 as this may impact other services. This Jira makes it configurable.

Assignee

Robin

Reporter

Robin

Labels

None

Fix versions

Affects versions

Priority

Medium
Configure