大数据之Hadoop集群中Yarn常用命令

2022-06-17 21:04:04 阅读：294 来源： 互联网

标签：06 hadoop yarn Hadoop application 1654496324557 Yarn 常用命令 hadoop103

Yarn状态的查询，除了可以在hadoop103:8088页面查看以外，还可以通过命令操作。常见的命令操作如下所示。

需求：执行WordCount案例，并通过Yarn命令查看任务运行情况。

# 启动集群
[hadoop@hadoop102 bin]$ myhadoop.sh start
# 运行wordcount案例
[hadoop@hadoop102 hadoop-3.1.3]$ hadoop jar share/hadoop/mapreduce/hadoop-mapreduce-examples-3.1.3.jar wordcount /input /output

yar application查看任务

1、列出所有Application：

[hadoop@hadoop102 hadoop-3.1.3]$ yarn application -list
2022-06-06 16:51:19,896 INFO client.RMProxy: Connecting to ResourceManager at hadoop103/192.168.10.103:8032
Total number of applications (application-types: [], states: [SUBMITTED, ACCEPTED, RUNNING] and tags: []):0
                Application-Id      Application-Name        Application-Type          User           Queue                   State              Final-State             Progress                        Tracking-URL

2、根据Application状态过滤：yarn application -list -appStates [ALL、NEW、NEW_SAVING、SUBMITTED、ACCEPTED、RUNNING、FINISHED、FAILED、KILLED]

[hadoop@hadoop102 hadoop-3.1.3]$ yarn application -list -appStates FINISHED
2022-06-06 16:53:49,886 INFO client.RMProxy: Connecting to ResourceManager at hadoop103/192.168.10.103:8032
Total number of applications (application-types: [], states: [FINISHED] and tags: []):2
                Application-Id      Application-Name        Application-Type          User           Queue                   State              Final-State             Progress                        Tracking-URL
application_1654496324557_0001            word count               MAPREDUCE        hadoop         default                FINISHED                   FAILED                 100% http://hadoop102:19888/jobhistory/job/job_1654496324557_0001
application_1654496324557_0002            word count               MAPREDUCE        hadoop         default                FINISHED                   FAILED                 100% http://hadoop102:19888/jobhistory/job/job_1654496324557_0002

2、Kill掉Application

[hadoop@hadoop102 hadoop-3.1.3]$ yarn application -kill application_1654496324557_0001
2022-06-06 16:57:34,082 INFO client.RMProxy: Connecting to ResourceManager at hadoop103/192.168.10.103:8032
Application application_1654496324557_0001 has already finished

yarn logs查看日志

1、查询Application日志：yarn logs -applicationId <ApplicationId>

[hadoop@hadoop102 hadoop-3.1.3]$ yarn logs -applicationId application_1654496324557_0001

2、查询Container日志：yarn logs -applicationId <ApplicationId> -containerId <ContainerId>

[hadoop@hadoop102 hadoop-3.1.3]$ yarn logs -applicationId application_1654496324557_0001 -containerId container_1654496324557_0001_01_000001

yarn applicationattempt查看尝试的任务

1、列出所有Application尝试的列表：yarn applicationattempt -list <ApplicationId>

[hadoop@hadoop102 hadoop-3.1.3]$ yarn applicationattempt -list application_1654496324557_0001
2022-06-06 17:05:10,914 INFO client.RMProxy: Connecting to ResourceManager at hadoop103/192.168.10.103:8032
Total number of application attempts :1
         ApplicationAttempt-Id                 State                        AM-Container-Id                            Tracking-URL
appattempt_1654496324557_0001_000001                FINISHED    container_1654496324557_0001_01_000001  http://hadoop103:8088/proxy/application_1654496324557_0001/

2、打印ApplicationAttempt状态：yarn applicationattempt -status <ApplicationAttemptId>

[hadoop@hadoop102 hadoop-3.1.3]$ yarn applicationattempt -status appattempt_1654496324557_0001_000001
2022-06-06 17:08:56,570 INFO client.RMProxy: Connecting to ResourceManager at hadoop103/192.168.10.103:8032
Application Attempt Report : 
        ApplicationAttempt-Id : appattempt_1654496324557_0001_000001
        State : FINISHED
        AMContainer : container_1654496324557_0001_01_000001
        Tracking-URL : http://hadoop103:8088/proxy/application_1654496324557_0001/
        RPC Port : 32771
        AM Host : hadoop103
        Diagnostics : Task failed task_1654496324557_0001_m_000000
Job failed as tasks failed. failedMaps:1 failedReduces:0 killedMaps:0 killedReduces: 0

yarn container查看容器

1、列出所有Container：yarn container -list <ApplicationAttemptId>

[hadoop@hadoop103 ~]$ yarn container -list appattempt_1654496324557_0004_000001
2022-06-06 17:14:40,313 INFO client.RMProxy: Connecting to ResourceManager at hadoop103/192.168.10.103:8032
Total number of containers :2
                  Container-Id            Start Time             Finish Time                   State                    Host       Node Http Address                                 LOG-URL
container_1654496324557_0004_01_000001  星期一 六月 06 17:13:55 +0800 2022                       N/A                 RUNNING         hadoop102:46450     http://hadoop102:8042   http://hadoop102:8042/node/containerlogs/container_1654496324557_0004_01_000001/hadoop
container_1654496324557_0004_01_000004  星期一 六月 06 17:14:38 +0800 2022                       N/A                 RUNNING         hadoop103:44058     http://hadoop103:8042   http://hadoop103:8042/node/containerlogs/container_1654496324557_0004_01_000004/hadoop

2、打印Container状态：yarn container -status <ContainerId>

[hadoop@hadoop103 ~]$ yarn container -status container_1654496324557_0006_01_000001
2022-06-06 17:20:51,357 INFO client.RMProxy: Connecting to ResourceManager at hadoop103/192.168.10.103:8032
Container Report : 
        Container-Id : container_1654496324557_0006_01_000001
        Start-Time : 1654507199188
        Finish-Time : 0
        State : RUNNING
        Execution-Type : GUARANTEED
        LOG-URL : http://hadoop103:8042/node/containerlogs/container_1654496324557_0006_01_000001/hadoop
        Host : hadoop103:44058
        NodeHttpAddress : http://hadoop103:8042
        Diagnostics : null

注：只有任务跑的途中才能看到container的状态

yarn node查看节点状态

[hadoop@hadoop103 ~]$ yarn node -list -all
2022-06-06 17:25:20,336 INFO client.RMProxy: Connecting to ResourceManager at hadoop103/192.168.10.103:8032
Total Nodes:3
         Node-Id             Node-State Node-Http-Address       Number-of-Running-Containers
 hadoop104:38945                RUNNING    hadoop104:8042                                  0
 hadoop103:44058                RUNNING    hadoop103:8042                                  0
 hadoop102:46450                RUNNING    hadoop102:8042                                  0

yarn rmadmin更新配置

加载队列配置：yarn rmadmin -refreshQueues

[hadoop@hadoop103 ~]$ yarn rmadmin -refreshQueues
2022-06-06 17:27:10,830 INFO client.RMProxy: Connecting to ResourceManager at hadoop103/192.168.10.103:8033

yarn queue查看队列

打印队列信息：yarn queue -status <QueueName>

[hadoop@hadoop103 ~]$ yarn queue -status default
2022-06-06 17:28:54,282 INFO client.RMProxy: Connecting to ResourceManager at hadoop103/192.168.10.103:8032
Queue Information : 
Queue Name : default
        State : RUNNING
        Capacity : 100.0%
        Current Capacity : .0%
        Maximum Capacity : 100.0%
        Default Node Label expression : <DEFAULT_PARTITION>
        Accessible Node Labels : *
        Preemption : disabled
        Intra-queue Preemption : disabled

下篇文章：[]() 相关文章： [大数据之Hadoop集群中MapReduce的Join操作](https://www.cnblogs.com/sw-code/p/16387043.html) [大数据之Hadoop集群的HDFS压力测试](https://www.cnblogs.com/sw-code/p/16387034.html)

标签：06,hadoop,yarn,Hadoop,application,1654496324557,Yarn,常用命令,hadoop103
来源： https://www.cnblogs.com/sw-code/p/16387038.html

本站声明： 1. iCode9 技术分享网（下文简称本站）提供的所有内容，仅供技术学习、探讨和分享；
2. 关于本站的所有留言、评论、转载及引用，纯属内容发起人的个人观点，与本站观点和立场无关；
3. 关于本站的所有言论和文字，纯属内容发起人的个人观点，与本站观点和立场无关；
4. 本站文章均是网友提供，不完全保证技术分享内容的完整性、准确性、时效性、风险性和版权归属；如您发现该文章侵犯了您的权益，可联系我们第一时间进行删除；
5. 本站为非盈利性的个人网站，所有内容不会用来进行牟利，也不会利用任何形式的广告来间接获益，纯粹是为了广大技术爱好者提供技术内容和技术思想的分享性交流网站。

ICode9