hadoop集群在进行wordcount运行时而成功时而错误
·
使用命令:
hadoop jar share/hadoop/mapreduce/hadoop-mapreduce-examples-3.1.3.jar wordcount /wcinput /wcoutput
出错时:
[2023-02-07 17:24:08.522]Container killed on request. Exit code is 143
[2023-02-07 17:24:08.950]Container exited with a non-zero exit code 143.
2023-02-07 17:24:22,063 INFO mapreduce.Job: Task Id : attempt_1675759968621_0007_m_000000_1, Status : FAILED
[2023-02-07 17:24:18.892]Container [pid=127909,containerID=container_1675759968621_0007_01_000003] is running 319953408B beyond the ‘VIRTUAL’ memory limit. Current usage: 240 MB of 1 GB physical memory used; 2.4 GB of 2.1 GB virtual memory used. Killing container.
1.多运行几遍代码。有可能会运行成功,几率问题
2.靠谱的方式:
在yarn-site.xml中添加如下配置信息,然后重启yarn
<property>
<description>Whether virtual memory limits will be enforced for
containers.</description>
<name>yarn.nodemanager.vmem-check-enabled</name>
<value>false</value>
</property>
不要忘记分发yarn-site.xml
xsync etc/hadoop/yarn-site.xml
更多推荐



所有评论(0)