Log yarn.log-aggregation-enable=true If false, the logs are kept on the nodes running the Node Manager. when aggregation is enabled, all the logs are aggregated and moved to the HDFS after the job completes.
Hadoop Cluster Performance Monitoring Ganglia, Nagios
Managing the Hadoop Cluster Using Vendor Tools Cloudera, Ambari