使用jindo-distcp-tool工具同步HDFS数据到COS失败

COS没有数据，在小文件数量特别多，成百万级就不行了，试了加参数也没有用，所有的jindo-distcp工具各个版本都无法解决，我裂开了啊

小于20G以下数据，且数据块比较大的情况下才可以同步过去

这是这个库碎片小文件的数量

COS没有数据，在小文件数量特别多，成百万级就不行了，试了加参数也没有用，所有的jindo-distcp工具各个版本都无法解决，我裂开了啊

我们支持过各种各样的客户，不存在无法解决的问题。
你能否把问题描述清楚一点，包括使用distcp、hdfs版本、操作系统及运行环境，distcp使用参数，或者更详细一点，最好能提供yarn log报错日志。

抱歉，是我的疏忽了，以下是相关信息：
distcp版本：4.6.7
hadoop版本：2.8.5
操作系统：linux
运行环境： jdk1.8
distcp命令：hadoop jar /root/jindo-distcp-tool-4.6.7.jar $args --src $src --dest $target --filters filter.txt --update --ignore --parallelism 20 【其中args是cos的密钥参数】
同步库大小：141.8G
文件数量：1277773
yarn log 日志片段：

它开了很多任务（都是hanoi_ods库的同步），但是我只执行一次，而且cos上无数据，以下是一个任务的日志详情：
`Log Type: syslog
Log Upload Time: Fri Jul 21 10:41:56 +0800 2023
Log Length: 171631
2023-07-21 10:41:38,790 INFO [main] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Created MRAppMaster for application appattempt_1657098760513_3771004_000001
2023-07-21 10:41:38,894 INFO [main] org.apache.hadoop.mapreduce.v2.app.MRAppMaster:
/************************************************************
[system properties]
os.name: Linux
os.version: 3.10.0-1062.9.1.el7.x86_64
java.home: /usr/local/jdk/jre
java.runtime.version: 1.8.0_191-b12
java.vendor: Oracle Corporation
java.version: 1.8.0_191
java.vm.name: Java HotSpot(TM) 64-Bit Server VM
java.class.path: /data/emr/yarn/local/usercache/hanoi/appcache/application_1657098760513_3771004/container_e27_1657098760513_3771004_01_000001:/usr/local/service/hadoop/etc/hadoop:/usr/local/service/hadoop/share/hadoop/common/hadoop-nfs-2.8.5.jar:/usr/local/service/hadoop/share/hadoop/common/hadoop-common-2.8.5-tests.jar:/usr/local/service/hadoop/share/hadoop/common/hadoop-common-2.8.5.jar:/usr/local/service/hadoop/share/hadoop/common/lib/joda-time-2.9.4.jar:/usr/local/service/hadoop/share/hadoop/common/lib/mockito-all-1.8.5.jar:/usr/local/service/hadoop/share/hadoop/common/lib/commons-collections-3.2.2.jar:/usr/local/service/hadoop/share/hadoop/common/lib/api-util-1.0.0-M20.jar:/usr/local/service/hadoop/share/hadoop/common/lib/jsch-0.1.54.jar:/usr/local/service/hadoop/share/hadoop/common/lib/stax-api-1.0-2.jar:/usr/local/service/hadoop/share/hadoop/common/lib/jersey-json-1.9.jar:/usr/local/service/hadoop/share/hadoop/common/lib/apacheds-kerberos-codec-2.0.0-M15.jar:/usr/local/service/hadoop/share/hadoop/common/lib/curator-recipes-2.7.1.jar:/usr/local/service/hadoop/share/hadoop/common/lib/jackson-mapper-asl-1.9.13.jar:/usr/local/service/hadoop/share/hadoop/common/lib/httpcore-4.4.4.jar:/usr/local/service/hadoop/share/hadoop/common/lib/jackson-xc-1.9.13.jar:/usr/local/service/hadoop/share/hadoop/common/lib/commons-net-3.1.jar:/usr/local/service/hadoop/share/hadoop/common/lib/commons-io-2.4.jar:/usr/local/service/hadoop/share/hadoop/common/lib/jetty-6.1.26.jar:/usr/local/service/hadoop/share/hadoop/common/lib/slf4j-log4j12-1.7.10.jar:/usr/local/service/hadoop/share/hadoop/common/lib/jaxb-api-2.2.2.jar:/usr/local/service/hadoop/share/hadoop/common/lib/jcip-annotations-1.0-1.jar:/usr/local/service/hadoop/share/hadoop/common/lib/gson-2.2.4.jar:/usr/local/service/hadoop/share/hadoop/common/lib/hadoop-annotations-2.8.5.jar:/usr/local/service/hadoop/share/hadoop/common/lib/hadoop-temrfs-1.0.6.jar:/usr/local/service/hadoop/share/hadoop/common/lib/hamcrest-core-1.3.jar:/usr/local/service/hadoop/share/hadoop/common/lib/jsp-api-2.1.jar:/usr/local/service/hadoop/share/hadoop/common/lib/curator-client-2.7.1.jar:/usr/local/service/hadoop/share/hadoop/common/lib/commons-beanutils-1.7.0.jar:/usr/local/service/hadoop/share/hadoop/common/lib/commons-math3-3.1.1.jar:/usr/local/service/hadoop/share/hadoop/common/lib/jackson-jaxrs-1.9.13.jar:/usr/local/service/hadoop/share/hadoop/common/lib/jetty-util-6.1.26.jar:/usr/local/service/hadoop/share/hadoop/common/lib/commons-digester-1.8.jar:/usr/local/service/hadoop/share/hadoop/common/lib/zookeeper-3.4.6.jar:/usr/local/service/hadoop/share/hadoop/common/lib/jaxb-impl-2.2.3-1.jar:/usr/local/service/hadoop/share/hadoop/common/lib/avro-1.7.4.jar:/usr/local/service/hadoop/share/hadoop/common/lib/apacheds-i18n-2.0.0-M15.jar:/usr/local/service/hadoop/share/hadoop/common/lib/curator-framework-2.7.1.jar:/usr/local/service/hadoop/share/hadoop/common/lib/java-xmlbuilder-0.4.jar:/usr/local/service/hadoop/share/hadoop/common/lib/jersey-core-1.9.jar:/usr/local/service/hadoop/share/hadoop/common/lib/commons-logging-1.1.3.jar:/usr/local/service/hadoop/share/hadoop/common/lib/commons-configuration-1.6.jar:/usr/local/service/hadoop/share/hadoop/common/lib/junit-4.11.jar:/usr/local/service/hadoop/share/hadoop/common/lib/servlet-api-2.5.jar:/usr/local/service/hadoop/share/hadoop/common/lib/jsr305-3.0.0.jar:/usr/local/service/hadoop/share/hadoop/common/lib/jackson-core-asl-1.9.13.jar:/usr/local/service/hadoop/share/hadoop/common/lib/api-asn1-api-1.0.0-M20.jar:/usr/local/service/hadoop/share/hadoop/common/lib/guava-11.0.2.jar:/usr/local/service/hadoop/share/hadoop/common/lib/commons-lang-2.6.jar:/usr/local/service/hadoop/share/hadoop/common/lib/xz-1.0.jar:/usr/local/service/hadoop/share/hadoop/common/lib/jets3t-0.9.0.jar:/usr/local/service/hadoop/share/hadoop/common/lib/chdfs_hadoop_plugin_network-2.0.jar:/usr/local/service/hadoop/share/hadoop/common/lib/log4j-1.2.17.jar:/usr/local/service/hadoop/share/hadoop/common/lib/netty-3.6.2.Final.jar:/usr/local/service/hadoop/share/hadoop/common/lib/jetty-sslengine-6.1.26.jar:/usr/local/service/hadoop/share/hadoop/common/lib/nimbus-jose-jwt-4.41.1.jar:/usr/local/service/hadoop/share/hadoop/common/lib/commons-codec-1.4.jar:/usr/local/service/hadoop/share/hadoop/common/lib/snappy-java-<IP_ADDRESS>.jar:/usr/local/service/hadoop/share/hadoop/common/lib/jettison-1.1.jar:/usr/local/service/hadoop/share/hadoop/common/lib/htrace-core4-4.0.1-incubating.jar:/usr/local/service/hadoop/share/hadoop/common/lib/commons-beanutils-core-1.8.0.jar:/usr/local/service/hadoop/share/hadoop/common/lib/xmlenc-0.52.jar:/usr/local/service/hadoop/share/hadoop/common/lib/activation-1.1.jar:/usr/local/service/hadoop/share/hadoop/common/lib/hadoop-lzo-0.4.21-SNAPSHOT.jar:/usr/local/service/hadoop/share/hadoop/common/lib/paranamer-2.3.jar:/usr/local/service/hadoop/share/hadoop/common/lib/commons-cli-1.2.jar:/usr/local/service/hadoop/share/hadoop/common/lib/commons-compress-1.4.1.jar:/usr/local/service/hadoop/share/hadoop/common/lib/asm-3.2.jar:/usr/local/service/hadoop/share/hadoop/common/lib/protobuf-java-2.5.0.jar:/usr/local/service/hadoop/share/hadoop/common/lib/hadoop-auth-2.8.5.jar:/usr/local/service/hadoop/share/hadoop/common/lib/httpclient-4.5.2.jar:/usr/local/service/hadoop/share/hadoop/common/lib/slf4j-api-1.7.10.jar:/usr/local/service/hadoop/share/hadoop/common/lib/json-smart-1.3.1.jar:/usr/local/service/hadoop/share/hadoop/common/lib/jersey-server-1.9.jar:/usr/local/service/hadoop/share/hadoop/hdfs/hadoop-hdfs-nfs-2.8.5.jar:/usr/local/service/hadoop/share/hadoop/hdfs/hadoop-hdfs-native-client-2.8.5-tests.jar:/usr/local/service/hadoop/share/hadoop/hdfs/hadoop-hdfs-native-client-2.8.5.jar:/usr/local/service/hadoop/share/hadoop/hdfs/hadoop-hdfs-2.8.5-tests.jar:/usr/local/service/hadoop/share/hadoop/hdfs/hadoop-hdfs-client-2.8.5-tests.jar:/usr/local/service/hadoop/share/hadoop/hdfs/hadoop-hdfs-2.8.5.jar:/usr/local/service/hadoop/share/hadoop/hdfs/hadoop-hdfs-client-2.8.5.jar:/usr/local/service/hadoop/share/hadoop/hdfs/lib/xml-apis-1.3.04.jar:/usr/local/service/hadoop/share/hadoop/hdfs/lib/leveldbjni-all-1.8.jar:/usr/local/service/hadoop/share/hadoop/hdfs/lib/jackson-mapper-asl-1.9.13.jar:/usr/local/service/hadoop/share/hadoop/hdfs/lib/commons-io-2.4.jar:/usr/local/service/hadoop/share/hadoop/hdfs/lib/jetty-6.1.26.jar:/usr/local/service/hadoop/share/hadoop/hdfs/lib/jetty-util-6.1.26.jar:/usr/local/service/hadoop/share/hadoop/hdfs/lib/jersey-core-1.9.jar:/usr/local/service/hadoop/share/hadoop/hdfs/lib/commons-logging-1.1.3.jar:/usr/local/service/hadoop/share/hadoop/hdfs/lib/servlet-api-2.5.jar:/usr/local/service/hadoop/share/hadoop/hdfs/lib/jsr305-3.0.0.jar:/usr/local/service/hadoop/share/hadoop/hdfs/lib/okio-1.4.0.jar:/usr/local/service/hadoop/share/hadoop/hdfs/lib/jackson-core-asl-1.9.13.jar:/usr/local/service/hadoop/share/hadoop/hdfs/lib/ranger-plugin-classloader-1.2.0.jar:/usr/local/service/hadoop/share/hadoop/hdfs/lib/guava-11.0.2.jar:/usr/local/service/hadoop/share/hadoop/hdfs/lib/ranger-hdfs-plugin-shim-1.2.0.jar:/usr/local/service/hadoop/share/hadoop/hdfs/lib/commons-lang-2.6.jar:/usr/local/service/hadoop/share/hadoop/hdfs/lib/okhttp-2.4.0.jar:/usr/local/service/hadoop/share/hadoop/hdfs/lib/log4j-1.2.17.jar:/usr/local/service/hadoop/share/hadoop/hdfs/lib/netty-3.6.2.Final.jar:/usr/local/service/hadoop/share/hadoop/hdfs/lib/commons-codec-1.4.jar:/usr/local/service/hadoop/share/hadoop/hdfs/lib/commons-daemon-1.0.13.jar:/usr/local/service/hadoop/share/hadoop/hdfs/lib/htrace-core4-4.0.1-incubating.jar:/usr/local/service/hadoop/share/hadoop/hdfs/lib/xmlenc-0.52.jar:/usr/local/service/hadoop/share/hadoop/hdfs/lib/netty-all-4.0.23.Final.jar:/usr/local/service/hadoop/share/hadoop/hdfs/lib/commons-cli-1.2.jar:/usr/local/service/hadoop/share/hadoop/hdfs/lib/asm-3.2.jar:/usr/local/service/hadoop/share/hadoop/hdfs/lib/protobuf-java-2.5.0.jar:/usr/local/service/hadoop/share/hadoop/hdfs/lib/hadoop-hdfs-client-2.8.5.jar:/usr/local/service/hadoop/share/hadoop/hdfs/lib/xercesImpl-2.9.1.jar:/usr/local/service/hadoop/share/hadoop/hdfs/lib/jersey-server-1.9.jar:/usr/local/service/hadoop/share/hadoop/yarn/hadoop-yarn-server-common-2.8.5.jar:/usr/local/service/hadoop/share/hadoop/yarn/hadoop-yarn-common-2.8.5.jar:/usr/local/service/hadoop/share/hadoop/yarn/hadoop-yarn-client-2.8.5.jar:/usr/local/service/hadoop/share/hadoop/yarn/spark-2.4.3-yarn-shuffle.jar:/usr/local/service/hadoop/share/hadoop/yarn/hadoop-yarn-applications-distributedshell-2.8.5.jar:/usr/local/service/hadoop/share/hadoop/yarn/hadoop-yarn-api-2.8.5.jar:/usr/local/service/hadoop/share/hadoop/yarn/hadoop-yarn-server-tests-2.8.5.jar:/usr/local/service/hadoop/share/hadoop/yarn/hadoop-yarn-server-timeline-pluginstorage-2.8.5.jar:/usr/local/service/hadoop/share/hadoop/yarn/hadoop-yarn-server-nodemanager-2.8.5.jar:/usr/local/service/hadoop/share/hadoop/yarn/hadoop-yarn-server-sharedcachemanager-2.8.5.jar:/usr/local/service/hadoop/share/hadoop/yarn/hadoop-yarn-registry-2.8.5.jar:/usr/local/service/hadoop/share/hadoop/yarn/hadoop-yarn-applications-unmanaged-am-launcher-2.8.5.jar:/usr/local/service/hadoop/share/hadoop/yarn/hadoop-yarn-server-applicationhistoryservice-2.8.5.jar:/usr/local/service/hadoop/share/hadoop/yarn/hadoop-yarn-server-resourcemanager-2.8.5.jar:/usr/local/service/hadoop/share/hadoop/yarn/hadoop-yarn-server-web-proxy-2.8.5.jar:/usr/local/service/hadoop/share/hadoop/yarn/lib/commons-collections-3.2.2.jar:/usr/local/service/hadoop/share/hadoop/yarn/lib/stax-api-1.0-2.jar:/usr/local/service/hadoop/share/hadoop/yarn/lib/curator-test-2.7.1.jar:/usr/local/service/hadoop/share/hadoop/yarn/lib/jersey-json-1.9.jar:/usr/local/service/hadoop/share/hadoop/yarn/lib/leveldbjni-all-1.8.jar:/usr/local/service/hadoop/share/hadoop/yarn/lib/jackson-mapper-asl-1.9.13.jar:/usr/local/service/hadoop/share/hadoop/yarn/lib/jackson-xc-1.9.13.jar:/usr/local/service/hadoop/share/hadoop/yarn/lib/commons-io-2.4.jar:/usr/local/service/hadoop/share/hadoop/yarn/lib/javassist-3.18.1-GA.jar:/usr/local/service/hadoop/share/hadoop/yarn/lib/jetty-6.1.26.jar:/usr/local/service/hadoop/share/hadoop/yarn/lib/jaxb-api-2.2.2.jar:/usr/local/service/hadoop/share/hadoop/yarn/lib/jersey-guice-1.9.jar:/usr/local/service/hadoop/share/hadoop/yarn/lib/curator-client-2.7.1.jar:/usr/local/service/hadoop/share/hadoop/yarn/lib/jackson-jaxrs-1.9.13.jar:/usr/local/service/hadoop/share/hadoop/yarn/lib/jetty-util-6.1.26.jar:/usr/local/service/hadoop/share/hadoop/yarn/lib/guice-3.0.jar:/usr/local/service/hadoop/share/hadoop/yarn/lib/zookeeper-3.4.6.jar:/usr/local/service/hadoop/share/hadoop/yarn/lib/jaxb-impl-2.2.3-1.jar:/usr/local/service/hadoop/share/hadoop/yarn/lib/zookeeper-3.4.6-tests.jar:/usr/local/service/hadoop/share/hadoop/yarn/lib/aopalliance-1.0.jar:/usr/local/service/hadoop/share/hadoop/yarn/lib/jersey-core-1.9.jar:/usr/local/service/hadoop/share/hadoop/yarn/lib/commons-logging-1.1.3.jar:/usr/local/service/hadoop/share/hadoop/yarn/lib/servlet-api-2.5.jar:/usr/local/service/hadoop/share/hadoop/yarn/lib/commons-math-2.2.jar:/usr/local/service/hadoop/share/hadoop/yarn/lib/jsr305-3.0.0.jar:/usr/local/service/hadoop/share/hadoop/yarn/lib/jackson-core-asl-1.9.13.jar:/usr/local/service/hadoop/share/hadoop/yarn/lib/jersey-client-1.9.jar:/usr/local/service/hadoop/share/hadoop/yarn/lib/ranger-plugin-classloader-1.2.0.jar:/usr/local/service/hadoop/share/hadoop/yarn/lib/guava-11.0.2.jar:/usr/local/service/hadoop/share/hadoop/yarn/lib/commons-lang-2.6.jar:/usr/local/service/hadoop/share/hadoop/yarn/lib/javax.inject-1.jar:/usr/local/service/hadoop/share/hadoop/yarn/lib/xz-1.0.jar:/usr/local/service/hadoop/share/hadoop/yarn/lib/guice-servlet-3.0.jar:/usr/local/service/hadoop/share/hadoop/yarn/lib/log4j-1.2.17.jar:/usr/local/service/hadoop/share/hadoop/yarn/lib/netty-3.6.2.Final.jar:/usr/local/service/hadoop/share/hadoop/yarn/lib/commons-codec-1.4.jar:/usr/local/service/hadoop/share/hadoop/yarn/lib/ranger-yarn-plugin-shim-1.2.0.jar:/usr/local/service/hadoop/share/hadoop/yarn/lib/jettison-1.1.jar:/usr/local/service/hadoop/share/hadoop/yarn/lib/activation-1.1.jar:/usr/local/service/hadoop/share/hadoop/yarn/lib/hadoop-lzo-0.4.21-SNAPSHOT.jar:/usr/local/service/hadoop/share/hadoop/yarn/lib/commons-cli-1.2.jar:/usr/local/service/hadoop/share/hadoop/yarn/lib/commons-compress-1.4.1.jar:/usr/local/service/hadoop/share/hadoop/yarn/lib/asm-3.2.jar:/usr/local/service/hadoop/share/hadoop/yarn/lib/protobuf-java-2.5.0.jar:/usr/local/service/hadoop/share/hadoop/yarn/lib/jersey-server-1.9.jar:/usr/local/service/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-client-app-2.8.5.jar:/usr/local/service/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-client-hs-2.8.5.jar:/usr/local/service/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-client-core-2.8.5.jar:/usr/local/service/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-client-hs-plugins-2.8.5.jar:/usr/local/service/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-client-shuffle-2.8.5.jar:/usr/local/service/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-2.8.5.jar:/usr/local/service/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-examples-2.8.5.jar:/usr/local/service/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-2.8.5-tests.jar:/usr/local/service/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-client-common-2.8.5.jar:/usr/local/service/hadoop/share/hadoop/mapreduce/lib/leveldbjni-all-1.8.jar:/usr/local/service/hadoop/share/hadoop/mapreduce/lib/jackson-mapper-asl-1.9.13.jar:/usr/local/service/hadoop/share/hadoop/mapreduce/lib/commons-io-2.4.jar:/usr/local/service/hadoop/share/hadoop/mapreduce/lib/jersey-guice-1.9.jar:/usr/local/service/hadoop/share/hadoop/mapreduce/lib/hadoop-annotations-2.8.5.jar:/usr/local/service/hadoop/share/hadoop/mapreduce/lib/hamcrest-core-1.3.jar:/usr/local/service/hadoop/share/hadoop/mapreduce/lib/guice-3.0.jar:/usr/local/service/hadoop/share/hadoop/mapreduce/lib/avro-1.7.4.jar:/usr/local/service/hadoop/share/hadoop/mapreduce/lib/aopalliance-1.0.jar:/usr/local/service/hadoop/share/hadoop/mapreduce/lib/jersey-core-1.9.jar:/usr/local/service/hadoop/share/hadoop/mapreduce/lib/junit-4.11.jar:/usr/local/service/hadoop/share/hadoop/mapreduce/lib/jackson-core-asl-1.9.13.jar:/usr/local/service/hadoop/share/hadoop/mapreduce/lib/javax.inject-1.jar:/usr/local/service/hadoop/share/hadoop/mapreduce/lib/xz-1.0.jar:/usr/local/service/hadoop/share/hadoop/mapreduce/lib/guice-servlet-3.0.jar:/usr/local/service/hadoop/share/hadoop/mapreduce/lib/log4j-1.2.17.jar:/usr/local/service/hadoop/share/hadoop/mapreduce/lib/netty-3.6.2.Final.jar:/usr/local/service/hadoop/share/hadoop/mapreduce/lib/snappy-java-<IP_ADDRESS>.jar:/usr/local/service/hadoop/share/hadoop/mapreduce/lib/paranamer-2.3.jar:/usr/local/service/hadoop/share/hadoop/mapreduce/lib/commons-compress-1.4.1.jar:/usr/local/service/hadoop/share/hadoop/mapreduce/lib/asm-3.2.jar:/usr/local/service/hadoop/share/hadoop/mapreduce/lib/protobuf-java-2.5.0.jar:/usr/local/service/hadoop/share/hadoop/mapreduce/lib/jersey-server-1.9.jar:job.jar/job.jar:job.jar/classes/:job.jar/lib/*:/data/emr/yarn/local/usercache/hanoi/appcache/application_1657098760513_3771004/container_e27_1657098760513_3771004_01_000001/job.jar
java.io.tmpdir: /data/emr/yarn/local/usercache/hanoi/appcache/application_1657098760513_3771004/container_e27_1657098760513_3771004_01_000001/tmp
user.dir: /data/emr/yarn/local/usercache/hanoi/appcache/application_1657098760513_3771004/container_e27_1657098760513_3771004_01_000001
user.name: hanoi
************************************************************/
2023-07-21 10:41:38,972 INFO [main] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Executing with tokens:
2023-07-21 10:41:38,972 INFO [main] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Kind: YARN_AM_RM_TOKEN, Service: , Ident: (appAttemptId { application_id { id: 3771004 cluster_timestamp:<PHONE_NUMBER>513 } attemptId: 1 } keyId: 794314708)
2023-07-21 10:41:39,048 INFO [main] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Kind: HDFS_DELEGATION_TOKEN, Service: ha-hdfs:HDFS43394, Ident: (HDFS_DELEGATION_TOKEN token 12130819 for hanoi)
2023-07-21 10:41:39,061 INFO [main] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Using mapred newApiCommitter.
2023-07-21 10:41:39,062 INFO [main] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: OutputCommitter set in config null
2023-07-21 10:41:39,094 INFO [main] org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter: File Output Committer Algorithm version is 1
2023-07-21 10:41:39,094 INFO [main] org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter: FileOutputCommitter skip cleanup _temporary folders under output directory:false, ignore cleanup failures: false
2023-07-21 10:41:39,262 INFO [Thread-4] com.aliyun.jindodata.jnative.NativeLogger: CommonJniLogging.cpp:34] Client logger level is info
2023-07-21 10:41:39,263 INFO [Thread-5] com.aliyun.jindodata.jnative.NativeLogger: JindoJniState.cpp:7] JindoJniState::prepareAfterInited() getting called
2023-07-21 10:41:39,279 INFO [Thread-4] com.aliyun.jindodata.jnative.NativeLogger: JcomMainBaseImpl.hpp:130] main doRun() return value True
2023-07-21 10:41:39,279 INFO [Thread-5] com.aliyun.jindodata.jnative.NativeLogger: JindoNative.cpp:42] Successfully initialized jni native
2023-07-21 10:41:39,612 INFO [main] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: OutputCommitter is org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter
2023-07-21 10:41:39,717 INFO [main] org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class org.apache.hadoop.mapreduce.jobhistory.EventType for class org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler
2023-07-21 10:41:39,718 INFO [main] org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class org.apache.hadoop.mapreduce.v2.app.job.event.JobEventType for class org.apache.hadoop.mapreduce.v2.app.MRAppMaster$JobEventDispatcher
2023-07-21 10:41:39,718 INFO [main] org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class org.apache.hadoop.mapreduce.v2.app.job.event.TaskEventType for class org.apache.hadoop.mapreduce.v2.app.MRAppMaster$TaskEventDispatcher
2023-07-21 10:41:39,719 INFO [main] org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class org.apache.hadoop.mapreduce.v2.app.job.event.TaskAttemptEventType for class org.apache.hadoop.mapreduce.v2.app.MRAppMaster$TaskAttemptEventDispatcher
2023-07-21 10:41:39,719 INFO [main] org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class org.apache.hadoop.mapreduce.v2.app.commit.CommitterEventType for class org.apache.hadoop.mapreduce.v2.app.commit.CommitterEventHandler
2023-07-21 10:41:39,720 INFO [main] org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class org.apache.hadoop.mapreduce.v2.app.speculate.Speculator$EventType for class org.apache.hadoop.mapreduce.v2.app.MRAppMaster$SpeculatorEventDispatcher
2023-07-21 10:41:39,720 INFO [main] org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class org.apache.hadoop.mapreduce.v2.app.rm.ContainerAllocator$EventType for class org.apache.hadoop.mapreduce.v2.app.MRAppMaster$ContainerAllocatorRouter
2023-07-21 10:41:39,721 INFO [main] org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncher$EventType for class org.apache.hadoop.mapreduce.v2.app.MRAppMaster$ContainerLauncherRouter
2023-07-21 10:41:39,747 INFO [main] org.apache.hadoop.mapreduce.v2.jobhistory.JobHistoryUtils: Default file system [hdfs://HDFS43394:8020]
2023-07-21 10:41:39,762 INFO [main] org.apache.hadoop.mapreduce.v2.jobhistory.JobHistoryUtils: Default file system [hdfs://HDFS43394:8020]
2023-07-21 10:41:39,778 INFO [main] org.apache.hadoop.mapreduce.v2.jobhistory.JobHistoryUtils: Default file system [hdfs://HDFS43394:8020]
2023-07-21 10:41:39,787 INFO [main] org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Emitting job history data to the timeline server is not enabled
2023-07-21 10:41:39,825 INFO [main] org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class org.apache.hadoop.mapreduce.v2.app.job.event.JobFinishEvent$Type for class org.apache.hadoop.mapreduce.v2.app.MRAppMaster$JobFinishEventHandler
2023-07-21 10:41:39,879 INFO [main] org.apache.hadoop.metrics2.impl.MetricsConfig: loaded properties from hadoop-metrics2.properties
2023-07-21 10:41:39,892 INFO [main] org.apache.hadoop.metrics2.impl.MetricsSinkAdapter: Sink ganglia started
2023-07-21 10:41:39,930 INFO [main] org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Scheduled Metric snapshot period at 60 second(s).
2023-07-21 10:41:39,930 INFO [main] org.apache.hadoop.metrics2.impl.MetricsSystemImpl: MRAppMaster metrics system started
2023-07-21 10:41:39,936 INFO [main] org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl: Adding job token for job_1657098760513_3771004 to jobTokenSecretManager
2023-07-21 10:41:40,017 INFO [main] org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl: Not uberizing job_1657098760513_3771004 because: not enabled; too many maps; too much RAM;
2023-07-21 10:41:40,030 INFO [main] org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl: Input size for job job_1657098760513_3771004 = 40. Number of splits = 20
2023-07-21 10:41:40,030 INFO [main] org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl: Number of reduces for job job_1657098760513_3771004 = 0
2023-07-21 10:41:40,030 INFO [main] org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl: job_1657098760513_3771004Job Transitioned from NEW to INITED
2023-07-21 10:41:40,031 INFO [main] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: MRAppMaster launching normal, non-uberized, multi-container job job_1657098760513_3771004.
2023-07-21 10:41:40,050 INFO [main] org.apache.hadoop.ipc.CallQueueManager: Using callQueue: class java.util.concurrent.LinkedBlockingQueue queueCapacity: 100 scheduler: class org.apache.hadoop.ipc.DefaultRpcScheduler
2023-07-21 10:41:40,056 INFO [Socket Reader #1 for port 40851] org.apache.hadoop.ipc.Server: Starting Socket Reader #1 for port 40851
2023-07-21 10:41:40,084 INFO [main] org.apache.hadoop.yarn.factories.impl.pb.RpcServerFactoryPBImpl: Adding protocol org.apache.hadoop.mapreduce.v2.api.MRClientProtocolPB to the server
2023-07-21 10:41:40,106 INFO [IPC Server Responder] org.apache.hadoop.ipc.Server: IPC Server Responder: starting
2023-07-21 10:41:40,106 INFO [IPC Server listener on 40851] org.apache.hadoop.ipc.Server: IPC Server listener on 40851: starting
2023-07-21 10:41:40,107 INFO [main] org.apache.hadoop.mapreduce.v2.app.client.MRClientService: Instantiated MRClientService at /<IP_ADDRESS>:40851
2023-07-21 10:41:40,162 INFO [main] org.mortbay.log: Logging to org.slf4j.impl.Log4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog
2023-07-21 10:41:40,168 INFO [main] org.apache.hadoop.security.authentication.server.AuthenticationFilter: Unable to initialize FileSignerSecretProvider, falling back to use random secrets.
2023-07-21 10:41:40,172 INFO [main] org.apache.hadoop.http.HttpRequestLog: Http request log for http.requests.mapreduce is not defined
2023-07-21 10:41:40,176 INFO [main] org.apache.hadoop.http.HttpServer2: Added global filter 'safety' (class=org.apache.hadoop.http.HttpServer2$QuotingInputFilter)
2023-07-21 10:41:40,204 INFO [main] org.apache.hadoop.http.HttpServer2: Added filter AM_PROXY_FILTER (class=org.apache.hadoop.yarn.server.webproxy.amfilter.AmIpFilter) to context mapreduce
2023-07-21 10:41:40,204 INFO [main] org.apache.hadoop.http.HttpServer2: Added filter AM_PROXY_FILTER (class=org.apache.hadoop.yarn.server.webproxy.amfilter.AmIpFilter) to context static
2023-07-21 10:41:40,206 INFO [main] org.apache.hadoop.http.HttpServer2: adding path spec: /mapreduce/*
2023-07-21 10:41:40,206 INFO [main] org.apache.hadoop.http.HttpServer2: adding path spec: /ws/*
2023-07-21 10:41:40,429 INFO [main] org.apache.hadoop.yarn.webapp.WebApps: Registered webapp guice modules
2023-07-21 10:41:40,430 INFO [main] org.apache.hadoop.http.HttpServer2: Jetty bound to port 35821
2023-07-21 10:41:40,430 INFO [main] org.mortbay.log: jetty-6.1.26
2023-07-21 10:41:40,454 INFO [main] org.mortbay.log: Extract jar:file:/usr/local/service/hadoop/share/hadoop/yarn/hadoop-yarn-common-2.8.5.jar!/webapps/mapreduce to /data/emr/yarn/local/usercache/hanoi/appcache/application_1657098760513_3771004/container_e27_1657098760513_3771004_01_000001/tmp/Jetty_0_0_0_0_35821_mapreduce____7qugp0/webapp
2023-07-21 10:41:41,178 INFO [main] org.mortbay.log: Started HttpServer2$SelectChannelConnectorWithSafeStartup@<IP_ADDRESS>:35821
2023-07-21 10:41:41,178 INFO [main] org.apache.hadoop.yarn.webapp.WebApps: Web app mapreduce started at 35821
2023-07-21 10:41:41,181 INFO [main] org.apache.hadoop.ipc.CallQueueManager: Using callQueue: class java.util.concurrent.LinkedBlockingQueue queueCapacity: 3000 scheduler: class org.apache.hadoop.ipc.DefaultRpcScheduler
2023-07-21 10:41:41,182 INFO [Socket Reader #1 for port 43574] org.apache.hadoop.ipc.Server: Starting Socket Reader #1 for port 43574
2023-07-21 10:41:41,211 INFO [IPC Server Responder] org.apache.hadoop.ipc.Server: IPC Server Responder: starting
2023-07-21 10:41:41,211 INFO [IPC Server listener on 43574] org.apache.hadoop.ipc.Server: IPC Server listener on 43574: starting
2023-07-21 10:41:41,225 INFO [main] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerRequestor: nodeBlacklistingEnabled:true
2023-07-21 10:41:41,225 INFO [main] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerRequestor: maxTaskFailuresPerNode is 3
2023-07-21 10:41:41,225 INFO [main] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerRequestor: blacklistDisablePercent is 33
2023-07-21 10:41:41,285 INFO [main] org.apache.hadoop.mapreduce.v2.app.rm.RMCommunicator: maxContainerCapability: <memory:117965, vCores:26>
2023-07-21 10:41:41,285 INFO [main] org.apache.hadoop.mapreduce.v2.app.rm.RMCommunicator: queue: root.default
2023-07-21 10:41:41,288 INFO [main] org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl: Upper limit on the thread pool size is 500
2023-07-21 10:41:41,288 INFO [main] org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl: The thread pool initial size is 10
2023-07-21 10:41:41,290 INFO [main] org.apache.hadoop.yarn.client.api.impl.ContainerManagementProtocolProxy: yarn.client.max-cached-nodemanagers-proxies : 0
2023-07-21 10:41:41,296 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl: job_1657098760513_3771004Job Transitioned from INITED to SETUP
2023-07-21 10:41:41,297 INFO [CommitterEvent Processor #0] org.apache.hadoop.mapreduce.v2.app.commit.CommitterEventHandler: Processing the event EventType: JOB_SETUP
2023-07-21 10:41:41,309 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl: job_1657098760513_3771004Job Transitioned from SETUP to RUNNING
2023-07-21 10:41:41,332 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskImpl: task_1657098760513_3771004_m_000000 Task Transitioned from NEW to SCHEDULED
2023-07-21 10:41:41,333 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskImpl: task_1657098760513_3771004_m_000001 Task Transitioned from NEW to SCHEDULED
2023-07-21 10:41:41,333 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskImpl: task_1657098760513_3771004_m_000002 Task Transitioned from NEW to SCHEDULED
2023-07-21 10:41:41,333 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskImpl: task_1657098760513_3771004_m_000003 Task Transitioned from NEW to SCHEDULED
2023-07-21 10:41:41,333 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskImpl: task_1657098760513_3771004_m_000004 Task Transitioned from NEW to SCHEDULED
2023-07-21 10:41:41,334 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskImpl: task_1657098760513_3771004_m_000005 Task Transitioned from NEW to SCHEDULED
2023-07-21 10:41:41,334 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskImpl: task_1657098760513_3771004_m_000006 Task Transitioned from NEW to SCHEDULED
2023-07-21 10:41:41,334 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskImpl: task_1657098760513_3771004_m_000007 Task Transitioned from NEW to SCHEDULED
2023-07-21 10:41:41,335 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskImpl: task_1657098760513_3771004_m_000008 Task Transitioned from NEW to SCHEDULED
2023-07-21 10:41:41,335 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskImpl: task_1657098760513_3771004_m_000009 Task Transitioned from NEW to SCHEDULED
2023-07-21 10:41:41,335 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskImpl: task_1657098760513_3771004_m_000010 Task Transitioned from NEW to SCHEDULED
2023-07-21 10:41:41,335 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskImpl: task_1657098760513_3771004_m_000011 Task Transitioned from NEW to SCHEDULED
2023-07-21 10:41:41,336 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskImpl: task_1657098760513_3771004_m_000012 Task Transitioned from NEW to SCHEDULED
2023-07-21 10:41:41,336 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskImpl: task_1657098760513_3771004_m_000013 Task Transitioned from NEW to SCHEDULED
2023-07-21 10:41:41,337 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskImpl: task_1657098760513_3771004_m_000014 Task Transitioned from NEW to SCHEDULED
2023-07-21 10:41:41,337 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskImpl: task_1657098760513_3771004_m_000015 Task Transitioned from NEW to SCHEDULED
2023-07-21 10:41:41,337 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskImpl: task_1657098760513_3771004_m_000016 Task Transitioned from NEW to SCHEDULED
2023-07-21 10:41:41,337 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskImpl: task_1657098760513_3771004_m_000017 Task Transitioned from NEW to SCHEDULED
2023-07-21 10:41:41,338 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskImpl: task_1657098760513_3771004_m_000018 Task Transitioned from NEW to SCHEDULED
2023-07-21 10:41:41,338 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskImpl: task_1657098760513_3771004_m_000019 Task Transitioned from NEW to SCHEDULED
2023-07-21 10:41:41,339 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1657098760513_3771004_m_000000_0 TaskAttempt Transitioned from NEW to UNASSIGNED
2023-07-21 10:41:41,339 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1657098760513_3771004_m_000001_0 TaskAttempt Transitioned from NEW to UNASSIGNED
2023-07-21 10:41:41,339 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1657098760513_3771004_m_000002_0 TaskAttempt Transitioned from NEW to UNASSIGNED
2023-07-21 10:41:41,339 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1657098760513_3771004_m_000003_0 TaskAttempt Transitioned from NEW to UNASSIGNED
2023-07-21 10:41:41,339 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1657098760513_3771004_m_000004_0 TaskAttempt Transitioned from NEW to UNASSIGNED
2023-07-21 10:41:41,340 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1657098760513_3771004_m_000005_0 TaskAttempt Transitioned from NEW to UNASSIGNED
2023-07-21 10:41:41,340 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1657098760513_3771004_m_000006_0 TaskAttempt Transitioned from NEW to UNASSIGNED
2023-07-21 10:41:41,340 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1657098760513_3771004_m_000007_0 TaskAttempt Transitioned from NEW to UNASSIGNED
2023-07-21 10:41:41,340 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1657098760513_3771004_m_000008_0 TaskAttempt Transitioned from NEW to UNASSIGNED
2023-07-21 10:41:41,340 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1657098760513_3771004_m_000009_0 TaskAttempt Transitioned from NEW to UNASSIGNED
2023-07-21 10:41:41,340 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1657098760513_3771004_m_000010_0 TaskAttempt Transitioned from NEW to UNASSIGNED
2023-07-21 10:41:41,340 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1657098760513_3771004_m_000011_0 TaskAttempt Transitioned from NEW to UNASSIGNED
2023-07-21 10:41:41,341 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1657098760513_3771004_m_000012_0 TaskAttempt Transitioned from NEW to UNASSIGNED
2023-07-21 10:41:41,341 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1657098760513_3771004_m_000013_0 TaskAttempt Transitioned from NEW to UNASSIGNED
2023-07-21 10:41:41,341 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1657098760513_3771004_m_000014_0 TaskAttempt Transitioned from NEW to UNASSIGNED
2023-07-21 10:41:41,342 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1657098760513_3771004_m_000015_0 TaskAttempt Transitioned from NEW to UNASSIGNED
2023-07-21 10:41:41,342 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1657098760513_3771004_m_000016_0 TaskAttempt Transitioned from NEW to UNASSIGNED
2023-07-21 10:41:41,342 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1657098760513_3771004_m_000017_0 TaskAttempt Transitioned from NEW to UNASSIGNED
2023-07-21 10:41:41,342 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1657098760513_3771004_m_000018_0 TaskAttempt Transitioned from NEW to UNASSIGNED
2023-07-21 10:41:41,342 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1657098760513_3771004_m_000019_0 TaskAttempt Transitioned from NEW to UNASSIGNED
2023-07-21 10:41:41,343 INFO [Thread-58] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: mapResourceRequest:<memory:4096, vCores:1>
2023-07-21 10:41:41,348 INFO [eventHandlingThread] org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Event Writer setup for JobId: job_1657098760513_3771004, File: hdfs://HDFS43394:8020/emr/hadoop-yarn/staging/hanoi/.staging/job_1657098760513_3771004/job_1657098760513_3771004_1.jhist
2023-07-21 10:41:41,957 INFO [Socket Reader #1 for port 40851] SecurityLogger.org.apache.hadoop.ipc.Server: Auth successful for hanoi/tjpm@EMR-R9RBK987 (auth:SIMPLE)
2023-07-21 10:41:41,962 INFO [Socket Reader #1 for port 40851] SecurityLogger.org.apache.hadoop.security.authorize.ServiceAuthorizationManager: Authorization successful for hanoi/tjpm@EMR-R9RBK987 (auth:TOKEN) for protocol=interface org.apache.hadoop.mapreduce.v2.api.MRClientProtocolPB
2023-07-21 10:41:42,287 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Before Scheduling: PendingReds:0 ScheduledMaps:20 ScheduledReds:0 AssignedMaps:0 AssignedReds:0 CompletedMaps:0 CompletedReds:0 ContAlloc:0 ContRel:0 HostLocal:0 RackLocal:0
2023-07-21 10:41:42,311 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerRequestor: getResources() for application_1657098760513_3771004: ask=3 release= 0 newContainers=0 finishedContainers=0 resourcelimit=<memory:1058816, vCores:233> knownNMs=9
2023-07-21 10:41:43,319 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Got allocated containers 9
2023-07-21 10:41:43,320 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Assigned container container_e27_1657098760513_3771004_01_000002 to attempt_1657098760513_3771004_m_000000_0
2023-07-21 10:41:43,322 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Assigned container container_e27_1657098760513_3771004_01_000003 to attempt_1657098760513_3771004_m_000001_0
2023-07-21 10:41:43,322 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Assigned container container_e27_1657098760513_3771004_01_000004 to attempt_1657098760513_3771004_m_000002_0
2023-07-21 10:41:43,322 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Assigned container container_e27_1657098760513_3771004_01_000005 to attempt_1657098760513_3771004_m_000003_0
2023-07-21 10:41:43,322 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Assigned container container_e27_1657098760513_3771004_01_000006 to attempt_1657098760513_3771004_m_000004_0
2023-07-21 10:41:43,322 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Assigned container container_e27_1657098760513_3771004_01_000007 to attempt_1657098760513_3771004_m_000005_0
2023-07-21 10:41:43,323 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Assigned container container_e27_1657098760513_3771004_01_000008 to attempt_1657098760513_3771004_m_000006_0
2023-07-21 10:41:43,323 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Assigned container container_e27_1657098760513_3771004_01_000009 to attempt_1657098760513_3771004_m_000007_0
2023-07-21 10:41:43,323 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Assigned container container_e27_1657098760513_3771004_01_000010 to attempt_1657098760513_3771004_m_000008_0
2023-07-21 10:41:43,323 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: After Scheduling: PendingReds:0 ScheduledMaps:11 ScheduledReds:0 AssignedMaps:9 AssignedReds:0 CompletedMaps:0 CompletedReds:0 ContAlloc:9 ContRel:0 HostLocal:0 RackLocal:9
2023-07-21 10:41:43,365 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: The job-jar file on the remote FS is hdfs://HDFS43394/emr/hadoop-yarn/staging/hanoi/.staging/job_1657098760513_3771004/job.jar
2023-07-21 10:41:43,368 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: The job-conf file on the remote FS is /emr/hadoop-yarn/staging/hanoi/.staging/job_1657098760513_3771004/job.xml
2023-07-21 10:41:43,368 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: Adding #1 tokens and #1 secret keys for NM use for launching container
2023-07-21 10:41:43,368 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: Size of containertokens_dob is 2
2023-07-21 10:41:43,369 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: Putting shuffle token in serviceData
2023-07-21 10:41:43,385 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1657098760513_3771004_m_000000_0 TaskAttempt Transitioned from UNASSIGNED to ASSIGNED
2023-07-21 10:41:43,388 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1657098760513_3771004_m_000001_0 TaskAttempt Transitioned from UNASSIGNED to ASSIGNED
2023-07-21 10:41:43,388 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1657098760513_3771004_m_000002_0 TaskAttempt Transitioned from UNASSIGNED to ASSIGNED
2023-07-21 10:41:43,389 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1657098760513_3771004_m_000003_0 TaskAttempt Transitioned from UNASSIGNED to ASSIGNED
2023-07-21 10:41:43,390 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1657098760513_3771004_m_000004_0 TaskAttempt Transitioned from UNASSIGNED to ASSIGNED
2023-07-21 10:41:43,390 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1657098760513_3771004_m_000005_0 TaskAttempt Transitioned from UNASSIGNED to ASSIGNED
2023-07-21 10:41:43,391 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1657098760513_3771004_m_000006_0 TaskAttempt Transitioned from UNASSIGNED to ASSIGNED
2023-07-21 10:41:43,391 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1657098760513_3771004_m_000007_0 TaskAttempt Transitioned from UNASSIGNED to ASSIGNED
2023-07-21 10:41:43,392 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1657098760513_3771004_m_000008_0 TaskAttempt Transitioned from UNASSIGNED to ASSIGNED
2023-07-21 10:41:43,393 INFO [ContainerLauncher #0] org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl: Processing the event EventType: CONTAINER_REMOTE_LAUNCH for container container_e27_1657098760513_3771004_01_000002 taskAttempt attempt_1657098760513_3771004_m_000000_0
2023-07-21 10:41:43,393 INFO [ContainerLauncher #1] org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl: Processing the event EventType: CONTAINER_REMOTE_LAUNCH for container container_e27_1657098760513_3771004_01_000003 taskAttempt attempt_1657098760513_3771004_m_000001_0
2023-07-21 10:41:43,394 INFO [ContainerLauncher #2] org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl: Processing the event EventType: CONTAINER_REMOTE_LAUNCH for container container_e27_1657098760513_3771004_01_000004 taskAttempt attempt_1657098760513_3771004_m_000002_0
2023-07-21 10:41:43,394 INFO [ContainerLauncher #3] org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl: Processing the event EventType: CONTAINER_REMOTE_LAUNCH for container container_e27_1657098760513_3771004_01_000005 taskAttempt attempt_1657098760513_3771004_m_000003_0
2023-07-21 10:41:43,394 INFO [ContainerLauncher #4] org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl: Processing the event EventType: CONTAINER_REMOTE_LAUNCH for container container_e27_1657098760513_3771004_01_000006 taskAttempt attempt_1657098760513_3771004_m_000004_0
2023-07-21 10:41:43,394 INFO [ContainerLauncher #5] org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl: Processing the event EventType: CONTAINER_REMOTE_LAUNCH for container container_e27_1657098760513_3771004_01_000007 taskAttempt attempt_1657098760513_3771004_m_000005_0
2023-07-21 10:41:43,395 INFO [ContainerLauncher #6] org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl: Processing the event EventType: CONTAINER_REMOTE_LAUNCH for container container_e27_1657098760513_3771004_01_000008 taskAttempt attempt_1657098760513_3771004_m_000006_0
2023-07-21 10:41:43,395 INFO [ContainerLauncher #7] org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl: Processing the event EventType: CONTAINER_REMOTE_LAUNCH for container container_e27_1657098760513_3771004_01_000009 taskAttempt attempt_1657098760513_3771004_m_000007_0
2023-07-21 10:41:43,395 INFO [ContainerLauncher #8] org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl: Processing the event EventType: CONTAINER_REMOTE_LAUNCH for container container_e27_1657098760513_3771004_01_000010 taskAttempt attempt_1657098760513_3771004_m_000008_0
2023-07-21 10:41:43,397 INFO [ContainerLauncher #7] org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl: Launching attempt_1657098760513_3771004_m_000007_0
2023-07-21 10:41:43,397 INFO [ContainerLauncher #1] org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl: Launching attempt_1657098760513_3771004_m_000001_0
2023-07-21 10:41:43,397 INFO [ContainerLauncher #4] org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl: Launching attempt_1657098760513_3771004_m_000004_0
2023-07-21 10:41:43,397 INFO [ContainerLauncher #5] org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl: Launching attempt_1657098760513_3771004_m_000005_0
2023-07-21 10:41:43,397 INFO [ContainerLauncher #0] org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl: Launching attempt_1657098760513_3771004_m_000000_0
2023-07-21 10:41:43,397 INFO [ContainerLauncher #3] org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl: Launching attempt_1657098760513_3771004_m_000003_0
2023-07-21 10:41:43,397 INFO [ContainerLauncher #6] org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl: Launching attempt_1657098760513_3771004_m_000006_0
2023-07-21 10:41:43,397 INFO [ContainerLauncher #8] org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl: Launching attempt_1657098760513_3771004_m_000008_0
2023-07-21 10:41:43,397 INFO [ContainerLauncher #2] org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl: Launching attempt_1657098760513_3771004_m_000002_0
2023-07-21 10:41:43,459 INFO [ContainerLauncher #5] org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl: Shuffle port returned by ContainerManager for attempt_1657098760513_3771004_m_000005_0 : 13562
2023-07-21 10:41:43,459 INFO [ContainerLauncher #1] org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl: Shuffle port returned by ContainerManager for attempt_1657098760513_3771004_m_000001_0 : 13562
2023-07-21 10:41:43,459 INFO [ContainerLauncher #8] org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl: Shuffle port returned by ContainerManager for attempt_1657098760513_3771004_m_000008_0 : 13562
2023-07-21 10:41:43,459 INFO [ContainerLauncher #3] org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl: Shuffle port returned by ContainerManager for attempt_1657098760513_3771004_m_000003_0 : 13562
2023-07-21 10:41:43,459 INFO [ContainerLauncher #2] org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl: Shuffle port returned by ContainerManager for attempt_1657098760513_3771004_m_000002_0 : 13562
2023-07-21 10:41:43,459 INFO [ContainerLauncher #6] org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl: Shuffle port returned by ContainerManager for attempt_1657098760513_3771004_m_000006_0 : 13562
2023-07-21 10:41:43,459 INFO [ContainerLauncher #4] org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl: Shuffle port returned by ContainerManager for attempt_1657098760513_3771004_m_000004_0 : 13562
2023-07-21 10:41:43,459 INFO [ContainerLauncher #7] org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl: Shuffle port returned by ContainerManager for attempt_1657098760513_3771004_m_000007_0 : 13562
2023-07-21 10:41:43,459 INFO [ContainerLauncher #0] org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl: Shuffle port returned by ContainerManager for attempt_1657098760513_3771004_m_000000_0 : 13562
2023-07-21 10:41:43,460 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: TaskAttempt: [attempt_1657098760513_3771004_m_000007_0] using containerId: [container_e27_1657098760513_3771004_01_000009 on NM: [<IP_ADDRESS>:5006]
2023-07-21 10:41:43,463 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1657098760513_3771004_m_000007_0 TaskAttempt Transitioned from ASSIGNED to RUNNING
2023-07-21 10:41:43,463 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: TaskAttempt: [attempt_1657098760513_3771004_m_000000_0] using containerId: [container_e27_1657098760513_3771004_01_000002 on NM: [<IP_ADDRESS>:5006]
2023-07-21 10:41:43,463 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1657098760513_3771004_m_000000_0 TaskAttempt Transitioned from ASSIGNED to RUNNING
2023-07-21 10:41:43,463 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: TaskAttempt: [attempt_1657098760513_3771004_m_000002_0] using containerId: [container_e27_1657098760513_3771004_01_000004 on NM: [<IP_ADDRESS>:5006]
2023-07-21 10:41:43,463 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1657098760513_3771004_m_000002_0 TaskAttempt Transitioned from ASSIGNED to RUNNING
2023-07-21 10:41:43,464 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: TaskAttempt: [attempt_1657098760513_3771004_m_000004_0] using containerId: [container_e27_1657098760513_3771004_01_000006 on NM: [<IP_ADDRESS>:5006]
2023-07-21 10:41:43,464 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1657098760513_3771004_m_000004_0 TaskAttempt Transitioned from ASSIGNED to RUNNING
2023-07-21 10:41:43,464 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: TaskAttempt: [attempt_1657098760513_3771004_m_000005_0] using containerId: [container_e27_1657098760513_3771004_01_000007 on NM: [<IP_ADDRESS>:5006]
2023-07-21 10:41:43,464 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1657098760513_3771004_m_000005_0 TaskAttempt Transitioned from ASSIGNED to RUNNING
2023-07-21 10:41:43,464 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: TaskAttempt: [attempt_1657098760513_3771004_m_000008_0] using containerId: [container_e27_1657098760513_3771004_01_000010 on NM: [<IP_ADDRESS>:5006]
2023-07-21 10:41:43,464 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1657098760513_3771004_m_000008_0 TaskAttempt Transitioned from ASSIGNED to RUNNING
2023-07-21 10:41:43,465 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: TaskAttempt: [attempt_1657098760513_3771004_m_000006_0] using containerId: [container_e27_1657098760513_3771004_01_000008 on NM: [<IP_ADDRESS>:5006]
2023-07-21 10:41:43,465 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1657098760513_3771004_m_000006_0 TaskAttempt Transitioned from ASSIGNED to RUNNING
2023-07-21 10:41:43,465 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: TaskAttempt: [attempt_1657098760513_3771004_m_000003_0] using containerId: [container_e27_1657098760513_3771004_01_000005 on NM: [<IP_ADDRESS>:5006]
2023-07-21 10:41:43,465 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1657098760513_3771004_m_000003_0 TaskAttempt Transitioned from ASSIGNED to RUNNING
2023-07-21 10:41:43,465 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: TaskAttempt: [attempt_1657098760513_3771004_m_000001_0] using containerId: [container_e27_1657098760513_3771004_01_000003 on NM: [<IP_ADDRESS>:5006]
2023-07-21 10:41:43,465 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1657098760513_3771004_m_000001_0 TaskAttempt Transitioned from ASSIGNED to RUNNING
2023-07-21 10:41:43,466 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskImpl: task_1657098760513_3771004_m_000007 Task Transitioned from SCHEDULED to RUNNING
2023-07-21 10:41:43,466 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskImpl: task_1657098760513_3771004_m_000000 Task Transitioned from SCHEDULED to RUNNING
2023-07-21 10:41:43,466 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskImpl: task_1657098760513_3771004_m_000002 Task Transitioned from SCHEDULED to RUNNING
2023-07-21 10:41:43,466 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskImpl: task_1657098760513_3771004_m_000004 Task Transitioned from SCHEDULED to RUNNING
2023-07-21 10:41:43,466 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskImpl: task_1657098760513_3771004_m_000005 Task Transitioned from SCHEDULED to RUNNING
2023-07-21 10:41:43,466 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskImpl: task_1657098760513_3771004_m_000008 Task Transitioned from SCHEDULED to RUNNING
2023-07-21 10:41:43,466 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskImpl: task_1657098760513_3771004_m_000006 Task Transitioned from SCHEDULED to RUNNING
2023-07-21 10:41:43,466 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskImpl: task_1657098760513_3771004_m_000003 Task Transitioned from SCHEDULED to RUNNING
2023-07-21 10:41:43,466 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskImpl: task_1657098760513_3771004_m_000001 Task Transitioned from SCHEDULED to RUNNING
2023-07-21 10:41:44,326 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerRequestor: getResources() for application_1657098760513_3771004: ask=3 release= 0 newContainers=9 finishedContainers=0 resourcelimit=<memory:985088, vCores:215> knownNMs=9
2023-07-21 10:41:44,326 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Got allocated containers 9
2023-07-21 10:41:44,327 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Assigned container container_e27_1657098760513_3771004_01_000011 to attempt_1657098760513_3771004_m_000009_0
2023-07-21 10:41:44,327 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Assigned container container_e27_1657098760513_3771004_01_000012 to attempt_1657098760513_3771004_m_000010_0
2023-07-21 10:41:44,327 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Assigned container container_e27_1657098760513_3771004_01_000013 to attempt_1657098760513_3771004_m_000011_0
2023-07-21 10:41:44,327 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Assigned container container_e27_1657098760513_3771004_01_000014 to attempt_1657098760513_3771004_m_000012_0
2023-07-21 10:41:44,327 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Assigned container container_e27_1657098760513_3771004_01_000015 to attempt_1657098760513_3771004_m_000013_0
2023-07-21 10:41:44,327 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Assigned container container_e27_1657098760513_3771004_01_000016 to attempt_1657098760513_3771004_m_000014_0
2023-07-21 10:41:44,327 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Assigned container container_e27_1657098760513_3771004_01_000017 to attempt_1657098760513_3771004_m_000015_0
2023-07-21 10:41:44,327 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Assigned container container_e27_1657098760513_3771004_01_000018 to attempt_1657098760513_3771004_m_000016_0
2023-07-21 10:41:44,327 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Assigned container container_e27_1657098760513_3771004_01_000019 to attempt_1657098760513_3771004_m_000017_0
2023-07-21 10:41:44,327 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1657098760513_3771004_m_000009_0 TaskAttempt Transitioned from UNASSIGNED to ASSIGNED
2023-07-21 10:41:44,327 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: After Scheduling: PendingReds:0 ScheduledMaps:2 ScheduledReds:0 AssignedMaps:18 AssignedReds:0 CompletedMaps:0 CompletedReds:0 ContAlloc:18 ContRel:0 HostLocal:0 RackLocal:18
2023-07-21 10:41:44,328 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1657098760513_3771004_m_000010_0 TaskAttempt Transitioned from UNASSIGNED to ASSIGNED
2023-07-21 10:41:44,329 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1657098760513_3771004_m_000011_0 TaskAttempt Transitioned from UNASSIGNED to ASSIGNED
2023-07-21 10:41:44,329 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1657098760513_3771004_m_000012_0 TaskAttempt Transitioned from UNASSIGNED to ASSIGNED
2023-07-21 10:41:44,330 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1657098760513_3771004_m_000013_0 TaskAttempt Transitioned from UNASSIGNED to ASSIGNED
2023-07-21 10:41:44,331 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1657098760513_3771004_m_000014_0 TaskAttempt Transitioned from UNASSIGNED to ASSIGNED
2023-07-21 10:41:44,331 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1657098760513_3771004_m_000015_0 TaskAttempt Transitioned from UNASSIGNED to ASSIGNED
2023-07-21 10:41:44,332 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1657098760513_3771004_m_000016_0 TaskAttempt Transitioned from UNASSIGNED to ASSIGNED
2023-07-21 10:41:44,332 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1657098760513_3771004_m_000017_0 TaskAttempt Transitioned from UNASSIGNED to ASSIGNED
2023-07-21 10:41:44,333 INFO [ContainerLauncher #0] org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl: Processing the event EventType: CONTAINER_REMOTE_LAUNCH for container container_e27_1657098760513_3771004_01_000012 taskAttempt attempt_1657098760513_3771004_m_000010_0
2023-07-21 10:41:44,333 INFO [ContainerLauncher #6] org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl: Processing the event EventType: CONTAINER_REMOTE_LAUNCH for container container_e27_1657098760513_3771004_01_000016 taskAttempt attempt_1657098760513_3771004_m_000014_0
2023-07-21 10:41:44,333 INFO [ContainerLauncher #1] org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl: Processing the event EventType: CONTAINER_REMOTE_LAUNCH for container container_e27_1657098760513_3771004_01_000017 taskAttempt attempt_1657098760513_3771004_m_000015_0
2023-07-21 10:41:44,333 INFO [ContainerLauncher #3] org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl: Processing the event EventType: CONTAINER_REMOTE_LAUNCH for container container_e27_1657098760513_3771004_01_000018 taskAttempt attempt_1657098760513_3771004_m_000016_0
2023-07-21 10:41:44,333 INFO [ContainerLauncher #2] org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl: Processing the event EventType: CONTAINER_REMOTE_LAUNCH for container container_e27_1657098760513_3771004_01_000015 taskAttempt attempt_1657098760513_3771004_m_000013_0
2023-07-21 10:41:44,333 INFO [ContainerLauncher #8] org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl: Processing the event EventType: CONTAINER_REMOTE_LAUNCH for container container_e27_1657098760513_3771004_01_000014 taskAttempt attempt_1657098760513_3771004_m_000012_0
2023-07-21 10:41:44,333 INFO [ContainerLauncher #4] org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl: Processing the event EventType: CONTAINER_REMOTE_LAUNCH for container container_e27_1657098760513_3771004_01_000013 taskAttempt attempt_1657098760513_3771004_m_000011_0
2023-07-21 10:41:44,333 INFO [ContainerLauncher #2] org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl: Launching attempt_1657098760513_3771004_m_000013_0
2023-07-21 10:41:44,333 INFO [ContainerLauncher #4] org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl: Launching attempt_1657098760513_3771004_m_000011_0
2023-07-21 10:41:44,333 INFO [ContainerLauncher #3] org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl: Launching attempt_1657098760513_3771004_m_000016_0
2023-07-21 10:41:44,333 INFO [ContainerLauncher #1] org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl: Launching attempt_1657098760513_3771004_m_000015_0
2023-07-21 10:41:44,333 INFO [ContainerLauncher #7] org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl: Processing the event EventType: CONTAINER_REMOTE_LAUNCH for container container_e27_1657098760513_3771004_01_000019 taskAttempt attempt_1657098760513_3771004_m_000017_0
2023-07-21 10:41:44,333 INFO [ContainerLauncher #6] org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl: Launching attempt_1657098760513_3771004_m_000014_0
2023-07-21 10:41:44,333 INFO [ContainerLauncher #9] org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl: Processing the event EventType: CONTAINER_REMOTE_LAUNCH for container container_e27_1657098760513_3771004_01_000011 taskAttempt attempt_1657098760513_3771004_m_000009_0
2023-07-21 10:41:44,333 INFO [ContainerLauncher #0] org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl: Launching attempt_1657098760513_3771004_m_000010_0
2023-07-21 10:41:44,334 INFO [ContainerLauncher #9] org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl: Launching attempt_1657098760513_3771004_m_000009_0
2023-07-21 10:41:44,334 INFO [ContainerLauncher #7] org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl: Launching attempt_1657098760513_3771004_m_000017_0
2023-07-21 10:41:44,333 INFO [ContainerLauncher #8] org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl: Launching attempt_1657098760513_3771004_m_000012_0
2023-07-21 10:41:44,346 INFO [ContainerLauncher #2] org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl: Shuffle port returned by ContainerManager for attempt_1657098760513_3771004_m_000013_0 : 13562
2023-07-21 10:41:44,346 INFO [ContainerLauncher #8] org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl: Shuffle port returned by ContainerManager for attempt_1657098760513_3771004_m_000012_0 : 13562
2023-07-21 10:41:44,346 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: TaskAttempt: [attempt_1657098760513_3771004_m_000013_0] using containerId: [container_e27_1657098760513_3771004_01_000015 on NM: [<IP_ADDRESS>:5006]
2023-07-21 10:41:44,347 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1657098760513_3771004_m_000013_0 TaskAttempt Transitioned from ASSIGNED to RUNNING
2023-07-21 10:41:44,348 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: TaskAttempt: [attempt_1657098760513_3771004_m_000012_0] using containerId: [container_e27_1657098760513_3771004_01_000014 on NM: [<IP_ADDRESS>:5006]
2023-07-21 10:41:44,351 INFO [ContainerLauncher #9] org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl: Shuffle port returned by ContainerManager for attempt_1657098760513_3771004_m_000009_0 : 13562
2023-07-21 10:41:44,351 INFO [ContainerLauncher #7] org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl: Shuffle port returned by ContainerManager for attempt_1657098760513_3771004_m_000017_0 : 13562
2023-07-21 10:41:44,352 INFO [ContainerLauncher #0] org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl: Shuffle port returned by ContainerManager for attempt_1657098760513_3771004_m_000010_0 : 13562
2023-07-21 10:41:44,352 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1657098760513_3771004_m_000012_0 TaskAttempt Transitioned from ASSIGNED to RUNNING
2023-07-21 10:41:44,353 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskImpl: task_1657098760513_3771004_m_000013 Task Transitioned from SCHEDULED to RUNNING
2023-07-21 10:41:44,354 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: TaskAttempt: [attempt_1657098760513_3771004_m_000009_0] using containerId: [container_e27_1657098760513_3771004_01_000011 on NM: [<IP_ADDRESS>:5006]
2023-07-21 10:41:44,355 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1657098760513_3771004_m_000009_0 TaskAttempt Transitioned from ASSIGNED to RUNNING
2023-07-21 10:41:44,355 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: TaskAttempt: [attempt_1657098760513_3771004_m_000017_0] using containerId: [container_e27_1657098760513_3771004_01_000019 on NM: [<IP_ADDRESS>:5006]
2023-07-21 10:41:44,355 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1657098760513_3771004_m_000017_0 TaskAttempt Transitioned from ASSIGNED to RUNNING
2023-07-21 10:41:44,355 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: TaskAttempt: [attempt_1657098760513_3771004_m_000010_0] using containerId: [container_e27_1657098760513_3771004_01_000012 on NM: [<IP_ADDRESS>:5006]
2023-07-21 10:41:44,356 INFO [ContainerLauncher #6] org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl: Shuffle port returned by ContainerManager for attempt_1657098760513_3771004_m_000014_0 : 13562
2023-07-21 10:41:44,356 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1657098760513_3771004_m_000010_0 TaskAttempt Transitioned from ASSIGNED to RUNNING
2023-07-21 10:41:44,356 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskImpl: task_1657098760513_3771004_m_000012 Task Transitioned from SCHEDULED to RUNNING
2023-07-21 10:41:44,356 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskImpl: task_1657098760513_3771004_m_000009 Task Transitioned from SCHEDULED to RUNNING
2023-07-21 10:41:44,356 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskImpl: task_1657098760513_3771004_m_000017 Task Transitioned from SCHEDULED to RUNNING
2023-07-21 10:41:44,356 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskImpl: task_1657098760513_3771004_m_000010 Task Transitioned from SCHEDULED to RUNNING
2023-07-21 10:41:44,356 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: TaskAttempt: [attempt_1657098760513_3771004_m_000014_0] using containerId: [container_e27_1657098760513_3771004_01_000016 on NM: [<IP_ADDRESS>:5006]
2023-07-21 10:41:44,356 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1657098760513_3771004_m_000014_0 TaskAttempt Transitioned from ASSIGNED to RUNNING
2023-07-21 10:41:44,356 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskImpl: task_1657098760513_3771004_m_000014 Task Transitioned from SCHEDULED to RUNNING
2023-07-21 10:41:44,363 INFO [ContainerLauncher #1] org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl: Shuffle port returned by ContainerManager for attempt_1657098760513_3771004_m_000015_0 : 13562
2023-07-21 10:41:44,363 INFO [ContainerLauncher #3] org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl: Shuffle port returned by ContainerManager for attempt_1657098760513_3771004_m_000016_0 : 13562
2023-07-21 10:41:44,364 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: TaskAttempt: [attempt_1657098760513_3771004_m_000015_0] using containerId: [container_e27_1657098760513_3771004_01_000017 on NM: [<IP_ADDRESS>:5006]
2023-07-21 10:41:44,364 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1657098760513_3771004_m_000015_0 TaskAttempt Transitioned from ASSIGNED to RUNNING
2023-07-21 10:41:44,364 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: TaskAttempt: [attempt_1657098760513_3771004_m_000016_0] using containerId: [container_e27_1657098760513_3771004_01_000018 on NM: [<IP_ADDRESS>:5006]
2023-07-21 10:41:44,364 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1657098760513_3771004_m_000016_0 TaskAttempt Transitioned from ASSIGNED to RUNNING
2023-07-21 10:41:44,364 INFO [ContainerLauncher #4] org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl: Shuffle port returned by ContainerManager for attempt_1657098760513_3771004_m_000011_0 : 13562
2023-07-21 10:41:44,365 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskImpl: task_1657098760513_3771004_m_000015 Task Transitioned from SCHEDULED to RUNNING
2023-07-21 10:41:44,365 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskImpl: task_1657098760513_3771004_m_000016 Task Transitioned from SCHEDULED to RUNNING
2023-07-21 10:41:44,365 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: TaskAttempt: [attempt_1657098760513_3771004_m_000011_0] using containerId: [container_e27_1657098760513_3771004_01_000013 on NM: [<IP_ADDRESS>:5006]
2023-07-21 10:41:44,365 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1657098760513_3771004_m_000011_0 TaskAttempt Transitioned from ASSIGNED to RUNNING
2023-07-21 10:41:44,365 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskImpl: task_1657098760513_3771004_m_000011 Task Transitioned from SCHEDULED to RUNNING
2023-07-21 10:41:45,331 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerRequestor: getResources() for application_1657098760513_3771004: ask=3 release= 0 newContainers=9 finishedContainers=0 resourcelimit=<memory:948224, vCores:206> knownNMs=9
2023-07-21 10:41:45,331 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Got allocated containers 9
2023-07-21 10:41:45,331 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Assigned container container_e27_1657098760513_3771004_01_000020 to attempt_1657098760513_3771004_m_000018_0
2023-07-21 10:41:45,331 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Assigned container container_e27_1657098760513_3771004_01_000021 to attempt_1657098760513_3771004_m_000019_0
2023-07-21 10:41:45,332 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Releasing unassigned container Container: [ContainerId: container_e27_1657098760513_3771004_01_000022, Version: 0, NodeId: <IP_ADDRESS>:5006, NodeHttpAddress: <IP_ADDRESS>:5008, Resource: <memory:4096, vCores:1>, Priority: 20, Token: Token { kind: ContainerToken, service: <IP_ADDRESS>:5006 }, ]
2023-07-21 10:41:45,332 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Releasing unassigned container Container: [ContainerId: container_e27_1657098760513_3771004_01_000023, Version: 0, NodeId: <IP_ADDRESS>:5006, NodeHttpAddress: <IP_ADDRESS>:5008, Resource: <memory:4096, vCores:1>, Priority: 20, Token: Token { kind: ContainerToken, service: <IP_ADDRESS>:5006 }, ]
2023-07-21 10:41:45,332 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Releasing unassigned container Container: [ContainerId: container_e27_1657098760513_3771004_01_000024, Version: 0, NodeId: <IP_ADDRESS>:5006, NodeHttpAddress: <IP_ADDRESS>:5008, Resource: <memory:4096, vCores:1>, Priority: 20, Token: Token { kind: ContainerToken, service: <IP_ADDRESS>:5006 }, ]
2023-07-21 10:41:45,332 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Releasing unassigned container Container: [ContainerId: container_e27_1657098760513_3771004_01_000025, Version: 0, NodeId: <IP_ADDRESS>:5006, NodeHttpAddress: <IP_ADDRESS>:5008, Resource: <memory:4096, vCores:1>, Priority: 20, Token: Token { kind: ContainerToken, service: <IP_ADDRESS>:5006 }, ]
2023-07-21 10:41:45,332 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Releasing unassigned container Container: [ContainerId: container_e27_1657098760513_3771004_01_000026, Version: 0, NodeId: <IP_ADDRESS>:5006, NodeHttpAddress: <IP_ADDRESS>:5008, Resource: <memory:4096, vCores:1>, Priority: 20, Token: Token { kind: ContainerToken, service: <IP_ADDRESS>:5006 }, ]
2023-07-21 10:41:45,332 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1657098760513_3771004_m_000018_0 TaskAttempt Transitioned from UNASSIGNED to ASSIGNED
2023-07-21 10:41:45,332 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Releasing unassigned container Container: [ContainerId: container_e27_1657098760513_3771004_01_000027, Version: 0, NodeId: <IP_ADDRESS>:5006, NodeHttpAddress: <IP_ADDRESS>:5008, Resource: <memory:4096, vCores:1>, Priority: 20, Token: Token { kind: ContainerToken, service: <IP_ADDRESS>:5006 }, ]
2023-07-21 10:41:45,332 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Releasing unassigned container Container: [ContainerId: container_e27_1657098760513_3771004_01_000028, Version: 0, NodeId: <IP_ADDRESS>:5006, NodeHttpAddress: <IP_ADDRESS>:5008, Resource: <memory:4096, vCores:1>, Priority: 20, Token: Token { kind: ContainerToken, service: <IP_ADDRESS>:5006 }, ]
2023-07-21 10:41:45,332 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: After Scheduling: PendingReds:0 ScheduledMaps:0 ScheduledReds:0 AssignedMaps:20 AssignedReds:0 CompletedMaps:0 CompletedReds:0 ContAlloc:27 ContRel:7 HostLocal:0 RackLocal:20
2023-07-21 10:41:45,332 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1657098760513_3771004_m_000019_0 TaskAttempt Transitioned from UNASSIGNED to ASSIGNED
2023-07-21 10:41:45,333 INFO [ContainerLauncher #6] org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl: Processing the event EventType: CONTAINER_REMOTE_LAUNCH for container container_e27_1657098760513_3771004_01_000021 taskAttempt attempt_1657098760513_3771004_m_000019_0
2023-07-21 10:41:45,333 INFO [ContainerLauncher #5] org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl: Processing the event EventType: CONTAINER_REMOTE_LAUNCH for container container_e27_1657098760513_3771004_01_000020 taskAttempt attempt_1657098760513_3771004_m_000018_0
2023-07-21 10:41:45,333 INFO [ContainerLauncher #6] org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl: Launching attempt_1657098760513_3771004_m_000019_0
2023-07-21 10:41:45,333 INFO [ContainerLauncher #5] org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl: Launching attempt_1657098760513_3771004_m_000018_0
2023-07-21 10:41:45,342 INFO [ContainerLauncher #6] org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl: Shuffle port returned by ContainerManager for attempt_1657098760513_3771004_m_000019_0 : 13562
2023-07-21 10:41:45,344 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: TaskAttempt: [attempt_1657098760513_3771004_m_000019_0] using containerId: [container_e27_1657098760513_3771004_01_000021 on NM: [<IP_ADDRESS>:5006]
2023-07-21 10:41:45,344 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1657098760513_3771004_m_000019_0 TaskAttempt Transitioned from ASSIGNED to RUNNING
2023-07-21 10:41:45,344 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskImpl: task_1657098760513_3771004_m_000019 Task Transitioned from SCHEDULED to RUNNING
2023-07-21 10:41:45,345 INFO [ContainerLauncher #5] org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl: Shuffle port returned by ContainerManager for attempt_1657098760513_3771004_m_000018_0 : 13562
2023-07-21 10:41:45,345 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: TaskAttempt: [attempt_1657098760513_3771004_m_000018_0] using containerId: [container_e27_1657098760513_3771004_01_000020 on NM: [<IP_ADDRESS>:5006]
2023-07-21 10:41:45,345 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1657098760513_3771004_m_000018_0 TaskAttempt Transitioned from ASSIGNED to RUNNING
2023-07-21 10:41:45,345 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskImpl: task_1657098760513_3771004_m_000018 Task Transitioned from SCHEDULED to RUNNING
2023-07-21 10:41:46,189 INFO [Socket Reader #1 for port 43574] SecurityLogger.org.apache.hadoop.ipc.Server: Auth successful for job_1657098760513_3771004 (auth:SIMPLE)
2023-07-21 10:41:46,190 INFO [Socket Reader #1 for port 43574] SecurityLogger.org.apache.hadoop.ipc.Server: Auth successful for job_1657098760513_3771004 (auth:SIMPLE)
2023-07-21 10:41:46,190 INFO [Socket Reader #1 for port 43574] SecurityLogger.org.apache.hadoop.ipc.Server: Auth successful for job_1657098760513_3771004 (auth:SIMPLE)
2023-07-21 10:41:46,196 INFO [Socket Reader #1 for port 43574] SecurityLogger.org.apache.hadoop.ipc.Server: Auth successful for job_1657098760513_3771004 (auth:SIMPLE)
2023-07-21 10:41:46,197 INFO [Socket Reader #1 for port 43574] SecurityLogger.org.apache.hadoop.ipc.Server: Auth successful for job_1657098760513_3771004 (auth:SIMPLE)
2023-07-21 10:41:46,197 INFO [Socket Reader #1 for port 43574] SecurityLogger.org.apache.hadoop.security.authorize.ServiceAuthorizationManager: Authorization successful for job_1657098760513_3771004 (auth:TOKEN) for protocol=interface org.apache.hadoop.mapred.TaskUmbilicalProtocol
2023-07-21 10:41:46,202 INFO [Socket Reader #1 for port 43574] SecurityLogger.org.apache.hadoop.security.authorize.ServiceAuthorizationManager: Authorization successful for job_1657098760513_3771004 (auth:TOKEN) for protocol=interface org.apache.hadoop.mapred.TaskUmbilicalProtocol
2023-07-21 10:41:46,203 INFO [Socket Reader #1 for port 43574] SecurityLogger.org.apache.hadoop.security.authorize.ServiceAuthorizationManager: Authorization successful for job_1657098760513_3771004 (auth:TOKEN) for protocol=interface org.apache.hadoop.mapred.TaskUmbilicalProtocol
2023-07-21 10:41:46,204 INFO [Socket Reader #1 for port 43574] SecurityLogger.org.apache.hadoop.ipc.Server: Auth successful for job_1657098760513_3771004 (auth:SIMPLE)
2023-07-21 10:41:46,204 INFO [IPC Server handler 1 on 43574] org.apache.hadoop.mapred.TaskAttemptListenerImpl: JVM with ID : jvm_1657098760513_3771004_m_29686813949961 asked for a task
2023-07-21 10:41:46,205 INFO [IPC Server handler 1 on 43574] org.apache.hadoop.mapred.TaskAttemptListenerImpl: JVM with ID: jvm_1657098760513_3771004_m_29686813949961 given task: attempt_1657098760513_3771004_m_000007_0
2023-07-21 10:41:46,205 INFO [Socket Reader #1 for port 43574] SecurityLogger.org.apache.hadoop.ipc.Server: Auth successful for job_1657098760513_3771004 (auth:SIMPLE)
2023-07-21 10:41:46,205 INFO [IPC Server handler 4 on 43574] org.apache.hadoop.mapred.TaskAttemptListenerImpl: JVM with ID : jvm_1657098760513_3771004_m_29686813949959 asked for a task
2023-07-21 10:41:46,205 INFO [IPC Server handler 2 on 43574] org.apache.hadoop.mapred.TaskAttemptListenerImpl: JVM with ID : jvm_1657098760513_3771004_m_29686813949956 asked for a task
2023-07-21 10:41:46,206 INFO [IPC Server handler 4 on 43574] org.apache.hadoop.mapred.TaskAttemptListenerImpl: JVM with ID: jvm_1657098760513_3771004_m_29686813949959 given task: attempt_1657098760513_3771004_m_000005_0
2023-07-21 10:41:46,206 INFO [IPC Server handler 2 on 43574] org.apache.hadoop.mapred.TaskAttemptListenerImpl: JVM with ID: jvm_1657098760513_3771004_m_29686813949956 given task: attempt_1657098760513_3771004_m_000002_0
2023-07-21 10:41:46,208 INFO [Socket Reader #1 for port 43574] SecurityLogger.org.apache.hadoop.security.authorize.ServiceAuthorizationManager: Authorization successful for job_1657098760513_3771004 (auth:TOKEN) for protocol=interface org.apache.hadoop.mapred.TaskUmbilicalProtocol
2023-07-21 10:41:46,208 INFO [Socket Reader #1 for port 43574] SecurityLogger.org.apache.hadoop.security.authorize.ServiceAuthorizationManager: Authorization successful for job_1657098760513_3771004 (auth:TOKEN) for protocol=interface org.apache.hadoop.mapred.TaskUmbilicalProtocol
2023-07-21 10:41:46,208 INFO [IPC Server handler 3 on 43574] org.apache.hadoop.mapred.TaskAttemptListenerImpl: JVM with ID : jvm_1657098760513_3771004_m_29686813949960 asked for a task
2023-07-21 10:41:46,208 INFO [IPC Server handler 3 on 43574] org.apache.hadoop.mapred.TaskAttemptListenerImpl: JVM with ID: jvm_1657098760513_3771004_m_29686813949960 given task: attempt_1657098760513_3771004_m_000006_0
2023-07-21 10:41:46,208 INFO [IPC Server handler 5 on 43574] org.apache.hadoop.mapred.TaskAttemptListenerImpl: JVM with ID : jvm_1657098760513_3771004_m_29686813949966 asked for a task
2023-07-21 10:41:46,208 INFO [IPC Server handler 5 on 43574] org.apache.hadoop.mapred.TaskAttemptListenerImpl: JVM with ID: jvm_1657098760513_3771004_m_29686813949966 given task: attempt_1657098760513_3771004_m_000012_0
2023-07-21 10:41:46,209 INFO [Socket Reader #1 for port 43574] SecurityLogger.org.apache.hadoop.ipc.Server: Auth successful for job_1657098760513_3771004 (auth:SIMPLE)
2023-07-21 10:41:46,211 INFO [Socket Reader #1 for port 43574] SecurityLogger.org.apache.hadoop.security.authorize.ServiceAuthorizationManager: Authorization successful for job_1657098760513_3771004 (auth:TOKEN) for protocol=interface org.apache.hadoop.mapred.TaskUmbilicalProtocol
2023-07-21 10:41:46,212 INFO [IPC Server handler 0 on 43574] org.apache.hadoop.mapred.TaskAttemptListenerImpl: JVM with ID : jvm_1657098760513_3771004_m_29686813949965 asked for a task
2023-07-21 10:41:46,212 INFO [IPC Server handler 0 on 43574] org.apache.hadoop.mapred.TaskAttemptListenerImpl: JVM with ID: jvm_1657098760513_3771004_m_29686813949965 given task: attempt_1657098760513_3771004_m_000011_0
2023-07-21 10:41:46,212 INFO [Socket Reader #1 for port 43574] SecurityLogger.org.apache.hadoop.security.authorize.ServiceAuthorizationManager: Authorization successful for job_1657098760513_3771004 (auth:TOKEN) for protocol=interface org.apache.hadoop.mapred.TaskUmbilicalProtocol
2023-07-21 10:41:46,213 INFO [IPC Server handler 13 on 43574] org.apache.hadoop.mapred.TaskAttemptListenerImpl: JVM with ID : jvm_1657098760513_3771004_m_29686813949957 asked for a task
2023-07-21 10:41:46,213 INFO [IPC Server handler 13 on 43574] org.apache.hadoop.mapred.TaskAttemptListenerImpl: JVM with ID: jvm_1657098760513_3771004_m_29686813949957 given task: attempt_1657098760513_3771004_m_000003_0
2023-07-21 10:41:46,218 INFO [Socket Reader #1 for port 43574] SecurityLogger.org.apache.hadoop.security.authorize.ServiceAuthorizationManager: Authorization successful for job_1657098760513_3771004 (auth:TOKEN) for protocol=interface org.apache.hadoop.mapred.TaskUmbilicalProtocol
2023-07-21 10:41:46,218 INFO [IPC Server handler 1 on 43574] org.apache.hadoop.mapred.TaskAttemptListenerImpl: JVM with ID : jvm_1657098760513_3771004_m_29686813949967 asked for a task
2023-07-21 10:41:46,218 INFO [IPC Server handler 1 on 43574] org.apache.hadoop.mapred.TaskAttemptListenerImpl: JVM with ID: jvm_1657098760513_3771004_m_29686813949967 given task: attempt_1657098760513_3771004_m_000013_0
2023-07-21 10:41:46,220 INFO [Socket Reader #1 for port 43574] SecurityLogger.org.apache.hadoop.ipc.Server: Auth successful for job_1657098760513_3771004 (auth:SIMPLE)
2023-07-21 10:41:46,221 INFO [Socket Reader #1 for port 43574] SecurityLogger.org.apache.hadoop.ipc.Server: Auth successful for job_1657098760513_3771004 (auth:SIMPLE)
2023-07-21 10:41:46,227 INFO [Socket Reader #1 for port 43574] SecurityLogger.org.apache.hadoop.ipc.Server: Auth successful for job_1657098760513_3771004 (auth:SIMPLE)
2023-07-21 10:41:46,228 INFO [Socket Reader #1 for port 43574] SecurityLogger.org.apache.hadoop.security.authorize.ServiceAuthorizationManager: Authorization successful for job_1657098760513_3771004 (auth:TOKEN) for protocol=interface org.apache.hadoop.mapred.TaskUmbilicalProtocol
2023-07-21 10:41:46,228 INFO [IPC Server handler 2 on 43574] org.apache.hadoop.mapred.TaskAttemptListenerImpl: JVM with ID : jvm_1657098760513_3771004_m_29686813949968 asked for a task
2023-07-21 10:41:46,228 INFO [IPC Server handler 2 on 43574] org.apache.hadoop.mapred.TaskAttemptListenerImpl: JVM with ID: jvm_1657098760513_3771004_m_29686813949968 given task: attempt_1657098760513_3771004_m_000014_0
2023-07-21 10:41:46,229 INFO [Socket Reader #1 for port 43574] SecurityLogger.org.apache.hadoop.security.authorize.ServiceAuthorizationManager: Authorization successful for job_1657098760513_3771004 (auth:TOKEN) for protocol=interface org.apache.hadoop.mapred.TaskUmbilicalProtocol
2023-07-21 10:41:46,229 INFO [IPC Server handler 5 on 43574] org.apache.hadoop.mapred.TaskAttemptListenerImpl: JVM with ID : jvm_1657098760513_3771004_m_29686813949962 asked for a task
2023-07-21 10:41:46,230 INFO [IPC Server handler 5 on 43574] org.apache.hadoop.mapred.TaskAttemptListenerImpl: JVM with ID: jvm_1657098760513_3771004_m_29686813949962 given task: attempt_1657098760513_3771004_m_000008_0
2023-07-21 10:41:46,235 INFO [Socket Reader #1 for port 43574] SecurityLogger.org.apache.hadoop.security.authorize.ServiceAuthorizationManager: Authorization successful for job_1657098760513_3771004 (auth:TOKEN) for protocol=interface org.apache.hadoop.mapred.TaskUmbilicalProtocol
2023-07-21 10:41:46,235 INFO [IPC Server handler 4 on 43574] org.apache.hadoop.mapred.TaskAttemptListenerImpl: JVM with ID : jvm_1657098760513_3771004_m_29686813949969 asked for a task
2023-07-21 10:41:46,235 INFO [IPC Server handler 4 on 43574] org.apache.hadoop.mapred.TaskAttemptListenerImpl: JVM with ID: jvm_1657098760513_3771004_m_29686813949969 given task: attempt_1657098760513_3771004_m_000015_0
2023-07-21 10:41:46,237 INFO [Socket Reader #1 for port 43574] SecurityLogger.org.apache.hadoop.ipc.Server: Auth successful for job_1657098760513_3771004 (auth:SIMPLE)
2023-07-21 10:41:46,238 INFO [Socket Reader #1 for port 43574] SecurityLogger.org.apache.hadoop.ipc.Server: Auth successful for job_1657098760513_3771004 (auth:SIMPLE)
2023-07-21 10:41:46,242 INFO [Socket Reader #1 for port 43574] SecurityLogger.org.apache.hadoop.ipc.Server: Auth successful for job_1657098760513_3771004 (auth:SIMPLE)
2023-07-21 10:41:46,245 INFO [Socket Reader #1 for port 43574] SecurityLogger.org.apache.hadoop.security.authorize.ServiceAuthorizationManager: Authorization successful for job_1657098760513_3771004 (auth:TOKEN) for protocol=interface org.apache.hadoop.mapred.TaskUmbilicalProtocol
2023-07-21 10:41:46,245 INFO [IPC Server handler 3 on 43574] org.apache.hadoop.mapred.TaskAttemptListenerImpl: JVM with ID : jvm_1657098760513_3771004_m_29686813949958 asked for a task
2023-07-21 10:41:46,245 INFO [IPC Server handler 3 on 43574] org.apache.hadoop.mapred.TaskAttemptListenerImpl: JVM with ID: jvm_1657098760513_3771004_m_29686813949958 given task: attempt_1657098760513_3771004_m_000004_0
2023-07-21 10:41:46,246 INFO [Socket Reader #1 for port 43574] SecurityLogger.org.apache.hadoop.ipc.Server: Auth successful for job_1657098760513_3771004 (auth:SIMPLE)
2023-07-21 10:41:46,246 INFO [Socket Reader #1 for port 43574] SecurityLogger.org.apache.hadoop.security.authorize.ServiceAuthorizationManager: Authorization successful for job_1657098760513_3771004 (auth:TOKEN) for protocol=interface org.apache.hadoop.mapred.TaskUmbilicalProtocol
2023-07-21 10:41:46,246 INFO [IPC Server handler 7 on 43574] org.apache.hadoop.mapred.TaskAttemptListenerImpl: JVM with ID : jvm_1657098760513_3771004_m_29686813949971 asked for a task
2023-07-21 10:41:46,246 INFO [IPC Server handler 7 on 43574] org.apache.hadoop.mapred.TaskAttemptListenerImpl: JVM with ID: jvm_1657098760513_3771004_m_29686813949971 given task: attempt_1657098760513_3771004_m_000017_0
2023-07-21 10:41:46,250 INFO [Socket Reader #1 for port 43574] SecurityLogger.org.apache.hadoop.security.authorize.ServiceAuthorizationManager: Authorization successful for job_1657098760513_3771004 (auth:TOKEN) for protocol=interface org.apache.hadoop.mapred.TaskUmbilicalProtocol
2023-07-21 10:41:46,251 INFO [IPC Server handler 6 on 43574] org.apache.hadoop.mapred.TaskAttemptListenerImpl: JVM with ID : jvm_1657098760513_3771004_m_29686813949955 asked for a task
2023-07-21 10:41:46,251 INFO [IPC Server handler 6 on 43574] org.apache.hadoop.mapred.TaskAttemptListenerImpl: JVM with ID: jvm_1657098760513_3771004_m_29686813949955 given task: attempt_1657098760513_3771004_m_000001_0
2023-07-21 10:41:46,254 INFO [Socket Reader #1 for port 43574] SecurityLogger.org.apache.hadoop.security.authorize.ServiceAuthorizationManager: Authorization successful for job_1657098760513_3771004 (auth:TOKEN) for protocol=interface org.apache.hadoop.mapred.TaskUmbilicalProtocol
2023-07-21 10:41:46,254 INFO [IPC Server handler 8 on 43574] org.apache.hadoop.mapred.TaskAttemptListenerImpl: JVM with ID : jvm_1657098760513_3771004_m_29686813949964 asked for a task
2023-07-21 10:41:46,254 INFO [IPC Server handler 8 on 43574] org.apache.hadoop.mapred.TaskAttemptListenerImpl: JVM with ID: jvm_1657098760513_3771004_m_29686813949964 given task: attempt_1657098760513_3771004_m_000010_0
2023-07-21 10:41:46,256 INFO [Socket Reader #1 for port 43574] SecurityLogger.org.apache.hadoop.ipc.Server: Auth successful for job_1657098760513_3771004 (auth:SIMPLE)
2023-07-21 10:41:46,264 INFO [Socket Reader #1 for port 43574] SecurityLogger.org.apache.hadoop.security.authorize.ServiceAuthorizationManager: Authorization successful for job_1657098760513_3771004 (auth:TOKEN) for protocol=interface org.apache.hadoop.mapred.TaskUmbilicalProtocol
2023-07-21 10:41:46,265 INFO [IPC Server handler 10 on 43574] org.apache.hadoop.mapred.TaskAttemptListenerImpl: JVM with ID : jvm_1657098760513_3771004_m_29686813949970 asked for a task
2023-07-21 10:41:46,265 INFO [IPC Server handler 10 on 43574] org.apache.hadoop.mapred.TaskAttemptListenerImpl: JVM with ID: jvm_1657098760513_3771004_m_29686813949970 given task: attempt_1657098760513_3771004_m_000016_0
2023-07-21 10:41:46,271 INFO [Socket Reader #1 for port 43574] SecurityLogger.org.apache.hadoop.ipc.Server: Auth successful for job_1657098760513_3771004 (auth:SIMPLE)
2023-07-21 10:41:46,279 INFO [Socket Reader #1 for port 43574] SecurityLogger.org.apache.hadoop.security.authorize.ServiceAuthorizationManager: Authorization successful for job_1657098760513_3771004 (auth:TOKEN) for protocol=interface org.apache.hadoop.mapred.TaskUmbilicalProtocol
2023-07-21 10:41:46,279 INFO [IPC Server handler 0 on 43574] org.apache.hadoop.mapred.TaskAttemptListenerImpl: JVM with ID : jvm_1657098760513_3771004_m_29686813949963 asked for a task
2023-07-21 10:41:46,279 INFO [IPC Server handler 0 on 43574] org.apache.hadoop.mapred.TaskAttemptListenerImpl: JVM with ID: jvm_1657098760513_3771004_m_29686813949963 given task: attempt_1657098760513_3771004_m_000009_0
2023-07-21 10:41:46,290 INFO [Socket Reader #1 for port 43574] SecurityLogger.org.apache.hadoop.ipc.Server: Auth successful for job_1657098760513_3771004 (auth:SIMPLE)
2023-07-21 10:41:46,298 INFO [Socket Reader #1 for port 43574] SecurityLogger.org.apache.hadoop.security.authorize.ServiceAuthorizationManager: Authorization successful for job_1657098760513_3771004 (auth:TOKEN) for protocol=interface org.apache.hadoop.mapred.TaskUmbilicalProtocol
2023-07-21 10:41:46,299 INFO [IPC Server handler 9 on 43574] org.apache.hadoop.mapred.TaskAttemptListenerImpl: JVM with ID : jvm_1657098760513_3771004_m_29686813949954 asked for a task
2023-07-21 10:41:46,299 INFO [IPC Server handler 9 on 43574] org.apache.hadoop.mapred.TaskAttemptListenerImpl: JVM with ID: jvm_1657098760513_3771004_m_29686813949954 given task: attempt_1657098760513_3771004_m_000000_0
2023-07-21 10:41:46,337 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerRequestor: getResources() for application_1657098760513_3771004: ask=3 release= 7 newContainers=2 finishedContainers=7 resourcelimit=<memory:968704, vCores:211> knownNMs=9
2023-07-21 10:41:46,338 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Received completed container container_e27_1657098760513_3771004_01_000025
2023-07-21 10:41:46,338 ERROR [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Container complete event for unknown container container_e27_1657098760513_3771004_01_000025
2023-07-21 10:41:46,338 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Received completed container container_e27_1657098760513_3771004_01_000027
2023-07-21 10:41:46,338 ERROR [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Container complete event for unknown container container_e27_1657098760513_3771004_01_000027
2023-07-21 10:41:46,338 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Received completed container container_e27_1657098760513_3771004_01_000022
2023-07-21 10:41:46,338 ERROR [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Container complete event for unknown container container_e27_1657098760513_3771004_01_000022
2023-07-21 10:41:46,338 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Received completed container container_e27_1657098760513_3771004_01_000024
2023-07-21 10:41:46,338 ERROR [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Container complete event for unknown container container_e27_1657098760513_3771004_01_000024
2023-07-21 10:41:46,338 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Received completed container container_e27_1657098760513_3771004_01_000026
2023-07-21 10:41:46,338 ERROR [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Container complete event for unknown container container_e27_1657098760513_3771004_01_000026
2023-07-21 10:41:46,338 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Received completed container container_e27_1657098760513_3771004_01_000023
2023-07-21 10:41:46,338 ERROR [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Container complete event for unknown container container_e27_1657098760513_3771004_01_000023
2023-07-21 10:41:46,338 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Received completed container container_e27_1657098760513_3771004_01_000028
2023-07-21 10:41:46,338 ERROR [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Container complete event for unknown container container_e27_1657098760513_3771004_01_000028
2023-07-21 10:41:46,338 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Got allocated containers 2
2023-07-21 10:41:46,338 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Cannot assign container Container: [ContainerId: container_e27_1657098760513_3771004_01_000029, Version: 0, NodeId: <IP_ADDRESS>:5006, NodeHttpAddress: <IP_ADDRESS>:5008, Resource: <memory:4096, vCores:1>, Priority: 20, Token: Token { kind: ContainerToken, service: <IP_ADDRESS>:5006 }, ] for a map as either  container memory less than required <memory:4096, vCores:1> or no pending map tasks - maps.isEmpty=true
2023-07-21 10:41:46,338 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Cannot assign container Container: [ContainerId: container_e27_1657098760513_3771004_01_000030, Version: 0, NodeId: <IP_ADDRESS>:5006, NodeHttpAddress: <IP_ADDRESS>:5008, Resource: <memory:4096, vCores:1>, Priority: 20, Token: Token { kind: ContainerToken, service: <IP_ADDRESS>:5006 }, ] for a map as either  container memory less than required <memory:4096, vCores:1> or no pending map tasks - maps.isEmpty=true
2023-07-21 10:41:46,338 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: After Scheduling: PendingReds:0 ScheduledMaps:0 ScheduledReds:0 AssignedMaps:20 AssignedReds:0 CompletedMaps:0 CompletedReds:0 ContAlloc:29 ContRel:9 HostLocal:0 RackLocal:20
2023-07-21 10:41:46,369 INFO [Socket Reader #1 for port 43574] SecurityLogger.org.apache.hadoop.ipc.Server: Auth successful for job_1657098760513_3771004 (auth:SIMPLE)
2023-07-21 10:41:46,378 INFO [Socket Reader #1 for port 43574] SecurityLogger.org.apache.hadoop.security.authorize.ServiceAuthorizationManager: Authorization successful for job_1657098760513_3771004 (auth:TOKEN) for protocol=interface org.apache.hadoop.mapred.TaskUmbilicalProtocol
2023-07-21 10:41:46,379 INFO [IPC Server handler 11 on 43574] org.apache.hadoop.mapred.TaskAttemptListenerImpl: JVM with ID : jvm_1657098760513_3771004_m_29686813949973 asked for a task
2023-07-21 10:41:46,379 INFO [IPC Server handler 11 on 43574] org.apache.hadoop.mapred.TaskAttemptListenerImpl: JVM with ID: jvm_1657098760513_3771004_m_29686813949973 given task: attempt_1657098760513_3771004_m_000019_0
2023-07-21 10:41:46,399 INFO [Socket Reader #1 for port 43574] SecurityLogger.org.apache.hadoop.ipc.Server: Auth successful for job_1657098760513_3771004 (auth:SIMPLE)
2023-07-21 10:41:46,407 INFO [Socket Reader #1 for port 43574] SecurityLogger.org.apache.hadoop.security.authorize.ServiceAuthorizationManager: Authorization successful for job_1657098760513_3771004 (auth:TOKEN) for protocol=interface org.apache.hadoop.mapred.TaskUmbilicalProtocol
2023-07-21 10:41:46,408 INFO [IPC Server handler 12 on 43574] org.apache.hadoop.mapred.TaskAttemptListenerImpl: JVM with ID : jvm_1657098760513_3771004_m_29686813949972 asked for a task
2023-07-21 10:41:46,408 INFO [IPC Server handler 12 on 43574] org.apache.hadoop.mapred.TaskAttemptListenerImpl: JVM with ID: jvm_1657098760513_3771004_m_29686813949972 given task: attempt_1657098760513_3771004_m_000018_0
2023-07-21 10:41:47,093 INFO [IPC Server handler 14 on 43574] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt attempt_1657098760513_3771004_m_000007_0 is : 0.0
2023-07-21 10:41:47,093 INFO [IPC Server handler 17 on 43574] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt attempt_1657098760513_3771004_m_000006_0 is : 0.0
2023-07-21 10:41:47,125 INFO [IPC Server handler 16 on 43574] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt attempt_1657098760513_3771004_m_000013_0 is : 0.0
2023-07-21 10:41:47,128 INFO [IPC Server handler 19 on 43574] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt attempt_1657098760513_3771004_m_000003_0 is : 0.0
2023-07-21 10:41:47,130 INFO [IPC Server handler 18 on 43574] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Commit-pending state update from attempt_1657098760513_3771004_m_000007_0
2023-07-21 10:41:47,131 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1657098760513_3771004_m_000007_0 TaskAttempt Transitioned from RUNNING to COMMIT_PENDING
2023-07-21 10:41:47,131 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskImpl: attempt_1657098760513_3771004_m_000007_0 given a go for committing the task output.
2023-07-21 10:41:47,132 INFO [IPC Server handler 13 on 43574] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt attempt_1657098760513_3771004_m_000017_0 is : 0.0
2023-07-21 10:41:47,133 INFO [IPC Server handler 21 on 43574] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Commit go/no-go request from attempt_1657098760513_3771004_m_000007_0
2023-07-21 10:41:47,133 INFO [IPC Server handler 22 on 43574] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt attempt_1657098760513_3771004_m_000015_0 is : 0.0
2023-07-21 10:41:47,133 INFO [IPC Server handler 21 on 43574] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskImpl: Result of canCommit for attempt_1657098760513_3771004_m_000007_0:true
2023-07-21 10:41:47,133 INFO [IPC Server handler 15 on 43574] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Commit-pending state update from attempt_1657098760513_3771004_m_000006_0
2023-07-21 10:41:47,135 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1657098760513_3771004_m_000006_0 TaskAttempt Transitioned from RUNNING to COMMIT_PENDING
2023-07-21 10:41:47,135 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskImpl: attempt_1657098760513_3771004_m_000006_0 given a go for committing the task output.
2023-07-21 10:41:47,136 INFO [IPC Server handler 20 on 43574] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Commit go/no-go request from attempt_1657098760513_3771004_m_000006_0
2023-07-21 10:41:47,136 INFO [IPC Server handler 20 on 43574] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskImpl: Result of canCommit for attempt_1657098760513_3771004_m_000006_0:true
2023-07-21 10:41:47,138 INFO [IPC Server handler 23 on 43574] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt attempt_1657098760513_3771004_m_000011_0 is : 0.0
2023-07-21 10:41:47,141 INFO [IPC Server handler 24 on 43574] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt attempt_1657098760513_3771004_m_000012_0 is : 0.0
2023-07-21 10:41:47,142 INFO [IPC Server handler 25 on 43574] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt attempt_1657098760513_3771004_m_000005_0 is : 0.0
2023-07-21 10:41:47,148 INFO [IPC Server handler 26 on 43574] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt attempt_1657098760513_3771004_m_000002_0 is : 0.0
2023-07-21 10:41:47,154 INFO [IPC Server handler 27 on 43574] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt attempt_1657098760513_3771004_m_000004_0 is : 0.0
2023-07-21 10:41:47,160 INFO [IPC Server handler 28 on 43574] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Commit-pending state update from attempt_1657098760513_3771004_m_000013_0
2023-07-21 10:41:47,161 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1657098760513_3771004_m_000013_0 TaskAttempt Transitioned from RUNNING to COMMIT_PENDING
2023-07-21 10:41:47,161 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskImpl: attempt_1657098760513_3771004_m_000013_0 given a go for committing the task output.
2023-07-21 10:41:47,162 INFO [IPC Server handler 29 on 43574] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Commit go/no-go request from attempt_1657098760513_3771004_m_000013_0
2023-07-21 10:41:47,162 INFO [IPC Server handler 29 on 43574] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskImpl: Result of canCommit for attempt_1657098760513_3771004_m_000013_0:true
2023-07-21 10:41:47,165 INFO [IPC Server handler 1 on 43574] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Commit-pending state update from attempt_1657098760513_3771004_m_000003_0
2023-07-21 10:41:47,166 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1657098760513_3771004_m_000003_0 TaskAttempt Transitioned from RUNNING to COMMIT_PENDING
2023-07-21 10:41:47,166 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskImpl: attempt_1657098760513_3771004_m_000003_0 given a go for committing the task output.
2023-07-21 10:41:47,166 INFO [IPC Server handler 2 on 43574] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Commit go/no-go request from attempt_1657098760513_3771004_m_000003_0
2023-07-21 10:41:47,166 INFO [IPC Server handler 2 on 43574] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskImpl: Result of canCommit for attempt_1657098760513_3771004_m_000003_0:true
2023-07-21 10:41:47,180 INFO [IPC Server handler 5 on 43574] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt attempt_1657098760513_3771004_m_000010_0 is : 0.0
2023-07-21 10:41:47,181 INFO [IPC Server handler 4 on 43574] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt attempt_1657098760513_3771004_m_000014_0 is : 0.0
2023-07-21 10:41:47,186 INFO [IPC Server handler 3 on 43574] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt attempt_1657098760513_3771004_m_000006_0 is : 1.0
2023-07-21 10:41:47,186 INFO [IPC Server handler 7 on 43574] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt attempt_1657098760513_3771004_m_000007_0 is : 1.0
2023-07-21 10:41:47,186 INFO [IPC Server handler 6 on 43574] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Commit-pending state update from attempt_1657098760513_3771004_m_000015_0
2023-07-21 10:41:47,186 INFO [IPC Server handler 8 on 43574] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Commit-pending state update from attempt_1657098760513_3771004_m_000011_0
2023-07-21 10:41:47,187 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1657098760513_3771004_m_000015_0 TaskAttempt Transitioned from RUNNING to COMMIT_PENDING
2023-07-21 10:41:47,187 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskImpl: attempt_1657098760513_3771004_m_000015_0 given a go for committing the task output.
2023-07-21 10:41:47,188 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1657098760513_3771004_m_000011_0 TaskAttempt Transitioned from RUNNING to COMMIT_PENDING
2023-07-21 10:41:47,188 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskImpl: attempt_1657098760513_3771004_m_000011_0 given a go for committing the task output.
2023-07-21 10:41:47,188 INFO [IPC Server handler 10 on 43574] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Commit-pending state update from attempt_1657098760513_3771004_m_000002_0
2023-07-21 10:41:47,190 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1657098760513_3771004_m_000002_0 TaskAttempt Transitioned from RUNNING to COMMIT_PENDING
2023-07-21 10:41:47,190 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskImpl: attempt_1657098760513_3771004_m_000002_0 given a go for committing the task output.
2023-07-21 10:41:47,190 INFO [IPC Server handler 10 on 43574] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Commit-pending state update from attempt_1657098760513_3771004_m_000012_0
2023-07-21 10:41:47,191 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1657098760513_3771004_m_000012_0 TaskAttempt Transitioned from RUNNING to COMMIT_PENDING
2023-07-21 10:41:47,191 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskImpl: attempt_1657098760513_3771004_m_000012_0 given a go for committing the task output.
2023-07-21 10:41:47,191 INFO [IPC Server handler 11 on 43574] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Commit-pending state update from attempt_1657098760513_3771004_m_000017_0
2023-07-21 10:41:47,191 INFO [IPC Server handler 9 on 43574] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt attempt_1657098760513_3771004_m_000008_0 is : 0.0
2023-07-21 10:41:47,191 INFO [IPC Server handler 12 on 43574] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Commit go/no-go request from attempt_1657098760513_3771004_m_000015_0
2023-07-21 10:41:47,192 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1657098760513_3771004_m_000017_0 TaskAttempt Transitioned from RUNNING to COMMIT_PENDING
2023-07-21 10:41:47,192 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskImpl: attempt_1657098760513_3771004_m_000017_0 given a go for committing the task output.
2023-07-21 10:41:47,192 INFO [IPC Server handler 12 on 43574] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskImpl: Result of canCommit for attempt_1657098760513_3771004_m_000015_0:true
2023-07-21 10:41:47,193 INFO [IPC Server handler 14 on 43574] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Commit-pending state update from attempt_1657098760513_3771004_m_000005_0
2023-07-21 10:41:47,193 INFO [IPC Server handler 12 on 43574] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Commit go/no-go request from attempt_1657098760513_3771004_m_000011_0
2023-07-21 10:41:47,193 INFO [IPC Server handler 12 on 43574] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskImpl: Result of canCommit for attempt_1657098760513_3771004_m_000011_0:true
2023-07-21 10:41:47,193 INFO [IPC Server handler 16 on 43574] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Commit-pending state update from attempt_1657098760513_3771004_m_000004_0
2023-07-21 10:41:47,193 INFO [IPC Server handler 19 on 43574] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Commit go/no-go request from attempt_1657098760513_3771004_m_000012_0
2023-07-21 10:41:47,194 INFO [IPC Server handler 18 on 43574] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Commit go/no-go request from attempt_1657098760513_3771004_m_000002_0
2023-07-21 10:41:47,194 INFO [IPC Server handler 19 on 43574] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskImpl: Result of canCommit for attempt_1657098760513_3771004_m_000012_0:true
2023-07-21 10:41:47,194 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1657098760513_3771004_m_000004_0 TaskAttempt Transitioned from RUNNING to COMMIT_PENDING
2023-07-21 10:41:47,194 INFO [IPC Server handler 13 on 43574] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Commit go/no-go request from attempt_1657098760513_3771004_m_000017_0
2023-07-21 10:41:47,194 INFO [IPC Server handler 18 on 43574] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskImpl: Result of canCommit for attempt_1657098760513_3771004_m_000002_0:true
2023-07-21 10:41:47,194 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskImpl: attempt_1657098760513_3771004_m_000004_0 given a go for committing the task output.
2023-07-21 10:41:47,195 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1657098760513_3771004_m_000005_0 TaskAttempt Transitioned from RUNNING to COMMIT_PENDING
2023-07-21 10:41:47,195 INFO [IPC Server handler 13 on 43574] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskImpl: Result of canCommit for attempt_1657098760513_3771004_m_000017_0:true
2023-07-21 10:41:47,196 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskImpl: attempt_1657098760513_3771004_m_000005_0 given a go for committing the task output.
2023-07-21 10:41:47,196 INFO [IPC Server handler 21 on 43574] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Commit go/no-go request from attempt_1657098760513_3771004_m_000004_0
2023-07-21 10:41:47,196 INFO [IPC Server handler 22 on 43574] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt attempt_1657098760513_3771004_m_000013_0 is : 1.0
2023-07-21 10:41:47,196 INFO [IPC Server handler 21 on 43574] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskImpl: Result of canCommit for attempt_1657098760513_3771004_m_000004_0:true
2023-07-21 10:41:47,196 INFO [IPC Server handler 15 on 43574] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Commit go/no-go request from attempt_1657098760513_3771004_m_000005_0
2023-07-21 10:41:47,196 INFO [IPC Server handler 15 on 43574] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskImpl: Result of canCommit for attempt_1657098760513_3771004_m_000005_0:true
2023-07-21 10:41:47,203 INFO [IPC Server handler 20 on 43574] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt attempt_1657098760513_3771004_m_000003_0 is : 1.0
2023-07-21 10:41:47,203 INFO [IPC Server handler 23 on 43574] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Done acknowledgement from attempt_1657098760513_3771004_m_000007_0
2023-07-21 10:41:47,203 INFO [IPC Server handler 24 on 43574] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Done acknowledgement from attempt_1657098760513_3771004_m_000006_0
2023-07-21 10:41:47,205 INFO [IPC Server handler 24 on 43574] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Done acknowledgement from attempt_1657098760513_3771004_m_000013_0
2023-07-21 10:41:47,207 INFO [IPC Server handler 26 on 43574] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt attempt_1657098760513_3771004_m_000001_0 is : 0.0
2023-07-21 10:41:47,207 INFO [IPC Server handler 27 on 43574] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Done acknowledgement from attempt_1657098760513_3771004_m_000003_0
2023-07-21 10:41:47,207 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1657098760513_3771004_m_000007_0 TaskAttempt Transitioned from COMMIT_PENDING to SUCCESS_FINISHING_CONTAINER
2023-07-21 10:41:47,208 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1657098760513_3771004_m_000006_0 TaskAttempt Transitioned from COMMIT_PENDING to SUCCESS_FINISHING_CONTAINER
2023-07-21 10:41:47,209 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1657098760513_3771004_m_000013_0 TaskAttempt Transitioned from COMMIT_PENDING to SUCCESS_FINISHING_CONTAINER
2023-07-21 10:41:47,211 INFO [IPC Server handler 28 on 43574] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt attempt_1657098760513_3771004_m_000016_0 is : 0.0
2023-07-21 10:41:47,212 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1657098760513_3771004_m_000003_0 TaskAttempt Transitioned from COMMIT_PENDING to SUCCESS_FINISHING_CONTAINER
2023-07-21 10:41:47,216 INFO [IPC Server handler 29 on 43574] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Commit-pending state update from attempt_1657098760513_3771004_m_000014_0
2023-07-21 10:41:47,217 INFO [IPC Server handler 1 on 43574] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Commit go/no-go request from attempt_1657098760513_3771004_m_000014_0
2023-07-21 10:41:47,218 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskImpl: Task succeeded with attempt attempt_1657098760513_3771004_m_000007_0
2023-07-21 10:41:47,219 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskImpl: task_1657098760513_3771004_m_000007 Task Transitioned from RUNNING to SUCCEEDED
2023-07-21 10:41:47,219 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskImpl: Task succeeded with attempt attempt_1657098760513_3771004_m_000006_0
2023-07-21 10:41:47,219 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskImpl: task_1657098760513_3771004_m_000006 Task Transitioned from RUNNING to SUCCEEDED
2023-07-21 10:41:47,219 INFO [IPC Server handler 2 on 43574] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Commit-pending state update from attempt_1657098760513_3771004_m_000010_0
2023-07-21 10:41:47,219 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskImpl: Task succeeded with attempt attempt_1657098760513_3771004_m_000013_0
2023-07-21 10:41:47,219 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskImpl: task_1657098760513_3771004_m_000013 Task Transitioned from RUNNING to SUCCEEDED
2023-07-21 10:41:47,219 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskImpl: Task succeeded with attempt attempt_1657098760513_3771004_m_000003_0
2023-07-21 10:41:47,219 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskImpl: task_1657098760513_3771004_m_000003 Task Transitioned from RUNNING to SUCCEEDED
2023-07-21 10:41:47,221 INFO [IPC Server handler 5 on 43574] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Commit go/no-go request from attempt_1657098760513_3771004_m_000010_0
2023-07-21 10:41:47,221 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1657098760513_3771004_m_000014_0 TaskAttempt Transitioned from RUNNING to COMMIT_PENDING
2023-07-21 10:41:47,223 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl: Num completed Tasks: 1
2023-07-21 10:41:47,223 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl: Num completed Tasks: 2
2023-07-21 10:41:47,223 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl: Num completed Tasks: 3
2023-07-21 10:41:47,223 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1657098760513_3771004_m_000010_0 TaskAttempt Transitioned from RUNNING to COMMIT_PENDING
2023-07-21 10:41:47,223 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl: Num completed Tasks: 4
2023-07-21 10:41:47,224 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskImpl: attempt_1657098760513_3771004_m_000014_0 given a go for committing the task output.
2023-07-21 10:41:47,224 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskImpl: attempt_1657098760513_3771004_m_000010_0 given a go for committing the task output.
2023-07-21 10:41:47,229 INFO [IPC Server handler 4 on 43574] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt attempt_1657098760513_3771004_m_000015_0 is : 1.0
2023-07-21 10:41:47,232 INFO [IPC Server handler 8 on 43574] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Done acknowledgement from attempt_1657098760513_3771004_m_000015_0
2023-07-21 10:41:47,233 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1657098760513_3771004_m_000015_0 TaskAttempt Transitioned from COMMIT_PENDING to SUCCESS_FINISHING_CONTAINER
2023-07-21 10:41:47,233 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskImpl: Task succeeded with attempt attempt_1657098760513_3771004_m_000015_0
2023-07-21 10:41:47,233 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskImpl: task_1657098760513_3771004_m_000015 Task Transitioned from RUNNING to SUCCEEDED
2023-07-21 10:41:47,233 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl: Num completed Tasks: 5
2023-07-21 10:41:47,234 INFO [IPC Server handler 6 on 43574] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt attempt_1657098760513_3771004_m_000012_0 is : 1.0
2023-07-21 10:41:47,234 INFO [IPC Server handler 0 on 43574] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt attempt_1657098760513_3771004_m_000004_0 is : 1.0
2023-07-21 10:41:47,236 INFO [IPC Server handler 10 on 43574] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt attempt_1657098760513_3771004_m_000005_0 is : 1.0
2023-07-21 10:41:47,237 INFO [IPC Server handler 11 on 43574] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt attempt_1657098760513_3771004_m_000011_0 is : 1.0
2023-07-21 10:41:47,241 INFO [IPC Server handler 17 on 43574] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt attempt_1657098760513_3771004_m_000002_0 is : 1.0
2023-07-21 10:41:47,241 INFO [IPC Server handler 12 on 43574] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Done acknowledgement from attempt_1657098760513_3771004_m_000004_0
2023-07-21 10:41:47,243 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1657098760513_3771004_m_000004_0 TaskAttempt Transitioned from COMMIT_PENDING to SUCCESS_FINISHING_CONTAINER
2023-07-21 10:41:47,243 INFO [IPC Server handler 11 on 43574] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Done acknowledgement from attempt_1657098760513_3771004_m_000012_0
2023-07-21 10:41:47,243 INFO [IPC Server handler 16 on 43574] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt attempt_1657098760513_3771004_m_000017_0 is : 1.0
2023-07-21 10:41:47,243 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskImpl: Task succeeded with attempt attempt_1657098760513_3771004_m_000004_0
2023-07-21 10:41:47,244 INFO [IPC Server handler 11 on 43574] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Commit-pending state update from attempt_1657098760513_3771004_m_000008_0
2023-07-21 10:41:47,244 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskImpl: task_1657098760513_3771004_m_000004 Task Transitioned from RUNNING to SUCCEEDED
2023-07-21 10:41:47,244 INFO [IPC Server handler 19 on 43574] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Done acknowledgement from attempt_1657098760513_3771004_m_000005_0
2023-07-21 10:41:47,244 INFO [IPC Server handler 18 on 43574] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Done acknowledgement from attempt_1657098760513_3771004_m_000011_0
2023-07-21 10:41:47,244 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1657098760513_3771004_m_000012_0 TaskAttempt Transitioned from COMMIT_PENDING to SUCCESS_FINISHING_CONTAINER
2023-07-21 10:41:47,244 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl: Num completed Tasks: 6
2023-07-21 10:41:47,244 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1657098760513_3771004_m_000008_0 TaskAttempt Transitioned from RUNNING to COMMIT_PENDING
2023-07-21 10:41:47,244 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1657098760513_3771004_m_000005_0 TaskAttempt Transitioned from COMMIT_PENDING to SUCCESS_FINISHING_CONTAINER
2023-07-21 10:41:47,245 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskImpl: Task succeeded with attempt attempt_1657098760513_3771004_m_000012_0
2023-07-21 10:41:47,245 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskImpl: task_1657098760513_3771004_m_000012 Task Transitioned from RUNNING to SUCCEEDED
2023-07-21 10:41:47,245 INFO [IPC Server handler 13 on 43574] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Done acknowledgement from attempt_1657098760513_3771004_m_000002_0
2023-07-21 10:41:47,245 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1657098760513_3771004_m_000011_0 TaskAttempt Transitioned from COMMIT_PENDING to SUCCESS_FINISHING_CONTAINER
2023-07-21 10:41:47,245 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskImpl: attempt_1657098760513_3771004_m_000008_0 given a go for committing the task output.
2023-07-21 10:41:47,245 INFO [IPC Server handler 21 on 43574] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Commit go/no-go request from attempt_1657098760513_3771004_m_000008_0
2023-07-21 10:41:47,245 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskImpl: Task succeeded with attempt attempt_1657098760513_3771004_m_000005_0
2023-07-21 10:41:47,245 INFO [IPC Server handler 15 on 43574] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Done acknowledgement from attempt_1657098760513_3771004_m_000017_0
2023-07-21 10:41:47,245 INFO [IPC Server handler 21 on 43574] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskImpl: Result of canCommit for attempt_1657098760513_3771004_m_000008_0:true
2023-07-21 10:41:47,245 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskImpl: task_1657098760513_3771004_m_000005 Task Transitioned from RUNNING to SUCCEEDED
2023-07-21 10:41:47,245 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl: Num completed Tasks: 7
2023-07-21 10:41:47,245 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1657098760513_3771004_m_000002_0 TaskAttempt Transitioned from COMMIT_PENDING to SUCCESS_FINISHING_CONTAINER
2023-07-21 10:41:47,245 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskImpl: Task succeeded with attempt attempt_1657098760513_3771004_m_000011_0
2023-07-21 10:41:47,246 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskImpl: task_1657098760513_3771004_m_000011 Task Transitioned from RUNNING to SUCCEEDED