Amazon EMR
      Back to home
      1. CNS Policies
      2. AWS Knowledge Base
      3. Amazon EMR

      AWS EMR Cluster Logging Disabled

      Risk Level: Medium

      Description: 

      This plugin ensures that Amazon S3 captures comprehensive log data from AWS Elastic MapReduce (EMR) clusters. EMR cluster logging should be enabled so that log files may be saved for troubleshooting.

      About the Service :

      Amazon EMR (Amazon Elastic MapReduce) is a managed cluster platform that makes it easier to run big data frameworks on AWS, such as Apache Hadoop and Apache Spark, to process and analyse large amounts of data. You may process data for analytics and business intelligence tasks using these frameworks and related open-source projects. Amazon EMR also allows you to convert and transport massive volumes of data into and out of other AWS data storage and databases, such as Amazon S3 and Amazon DynamoDB.

      Impact : 

      After the retention time expires, all EMR log files are automatically erased from the clusters. Elastic MapReduce uses this capability to upload log files from the cluster master instance(s) to Amazon S3 so that the logging data may be used afterwards for troubleshooting or compliance.

      Steps to reproduce :

      1. Log in to your AWS Management Console.
        https://console.aws.amazon.com/ 
      2. Navigate to the EMR Dashboard.
        https://console.aws.amazon.com/elasticmapreduce/ 
      3. In the left navigation panel, click on Clusters.
      4. Click on the cluster that you want to examine.
      5. In the Configuration Details section, check the Log URL. If there is no url mentioned that suggests that the cluster logging is disabled.
      6. Repeat the steps for other clusters as well.

      Steps for remediation :

      1. Log in to your AWS Management Console.
        https://console.aws.amazon.com/ 
      2. Navigate to the EMR Dashboard.
        https://console.aws.amazon.com/elasticmapreduce/ 
      3. In the left navigation panel, click on Clusters.
      4. Click on the cluster that you want to examine.
      5. In the Configuration Details section, check the Log URL. If there is no url mentioned that suggests that the cluster logging is disabled.
      6. Click on the Clone button on the top left corner, next select Yes and start the cloning process.
      7. Next, on the Create Cluster page, navigate to the General Settings and then check the Logging box.
      8. Click Next and once this cluster is up terminate the old cluster by clicking on Terminate.
      9. Repeat the steps for other clusters as well.

      References: