ICode9

精准搜索请尝试: 精确搜索
首页 > 其他分享> 文章详细

Hadoop常用操作

2022-01-25 17:34:09  阅读:200  来源: 互联网

标签:testtable 常用 hdfs hadoop dfs Hadoop test 操作 hbase


启动,关闭

D:\hadoop-2.8.3\hadoop-2.8.3\sbin>start-all.cmd

D:\hadoop-2.8.3\hadoop-2.8.3\sbin>stop-all.cmd

start-all.sh等价于start-dfs.sh + start-yarn.sh

单进程启动。

sbin/start-dfs.sh

---------------

    sbin/hadoop-daemons.sh --config .. --hostname .. start namenode ...
    sbin/hadoop-daemons.sh --config .. --hostname .. start datanode ...
    sbin/hadoop-daemons.sh --config .. --hostname .. start sescondarynamenode ...
    sbin/hadoop-daemons.sh --config .. --hostname .. start zkfc ...         //

sbin/start-yarn.sh
--------------  
    libexec/yarn-config.sh
    sbin/yarn-daemon.sh --config $YARN_CONF_DIR  start resourcemanager
    sbin/yarn-daemons.sh  --config $YARN_CONF_DIR  start nodemanager

web地址

http://localhost:50070/explorer.html#/

新增目录

创建新目录

hdfs dfs -mkdir -p /cmd_create_dir

将本地文件存储至hadoop

hdfs dfs -put ./demo_mvn.jar /test/

新建空文件

hdfs dfs -touchz /cmd_create_dir/empty.txt

某个文件重命名

hdfs dfs -mv /cmd_create_dir/empty.txt /cmd_create_dir/rename_empty.txt

修改目录

下载文件

D:\GITSPACE\demo\out\artifacts\demo_mvn_jar>hadoop dfs -get /test/test.xml C:\Users\chang\Desktop\fsdownload\security

DEPRECATED: Use of this script to execute hdfs command is deprecated.

Instead use the hdfs command for it.

D:\GITSPACE\demo\out\artifacts\demo_mvn_jar>hdfs dfs -get  /test/test.xml C:\Users\chang\Desktop\fsdownload\security

get: `C:/Users/chang/Desktop/fsdownload/security/test.xml': File exists

D:\GITSPACE\demo\out\artifacts\demo_mvn_jar>hdfs dfs -get  /test/test.xml C:\Users\chang\Desktop\fsdownload\

hdfs dfs -get /cmd_create_dir C:\Users\chang\Desktop\fsdownload\

将hadoop指定目录下所有内容保存为一个文件,同时down至本地

   hdfs dfs –getmerge /user /home/t

删除目录

hadoop fs -rm -r -skipTrash /path_to_file/file_name

D:\GITSPACE\demo\out\artifacts\demo_mvn_jar>hadoop fs -rm -r -skipTrash /test/word_output_json

Deleted /test/word_output_json

hadoop fs -rm -r /test/new2

将正在运行的hadoop作业kill掉

   hadoop job –kill  [job-id]

查看目录

hadoop fs -cat /test/word_output_json/part-r-00000

查看指定目录下内容

hdfs dfs -ls /

打开某个已存在文件

hdfs dfs -cat /test/test.xml

hdfs dfs -lsr /

HBase常用操作

D:\hbase-2.0.2-bin\hbase-2.0.2\bin>start-hbase.cmd

D:\hbase-2.0.2-bin\hbase-2.0.2\bin>hbase shell

status

create 'testtable','coln1'

list_namespace_tables 'default'

hbase(main):011:0> create_namespace 'testnamespace'

Took 0.2850 seconds

hbase(main):012:0> create 'testnamespace:testtable', 'coln1'

Created table testnamespace:testtable

Took 0.7770 seconds

=> Hbase::Table - testnamespace:testtable

hbase(main):013:0> list

新增数据

hbase(main):023:0> put "testtable", "row1", "coln1",'zhangsan'

Took 0.1300 seconds

hbase(main):024:0> put "testtable", "row2", "coln1",'lisi'

Took 0.0020 seconds

查询数据

hbase(main):027:0> get "testtable", "row1"

COLUMN                CELL

 coln1:               timestamp=1639382832908, value=lisi

1 row(s)

Took 0.0180 seconds

hbase(main):028:0> get "testtable", "coln1"

COLUMN                CELL

0 row(s)

Took 0.0040 seconds

hbase(main):029:0> get "testtable", "row1", "coln1"

COLUMN                CELL

 coln1:               timestamp=1639382832908, value=lisi

1 row(s)

Took 0.0040 seconds

删除数据

delete "testtable", "row1", "coln1"

deleteall "testtable", "row1"

hbase(main):071:0> disable "testtable"

Took 0.7690 seconds

hbase(main):072:0> drop "testtable"

Took 0.2570 seconds

hbase(main):073:0> drop "testtable1"

标签:testtable,常用,hdfs,hadoop,dfs,Hadoop,test,操作,hbase
来源: https://blog.csdn.net/u010479989/article/details/122688924

本站声明: 1. iCode9 技术分享网(下文简称本站)提供的所有内容,仅供技术学习、探讨和分享;
2. 关于本站的所有留言、评论、转载及引用,纯属内容发起人的个人观点,与本站观点和立场无关;
3. 关于本站的所有言论和文字,纯属内容发起人的个人观点,与本站观点和立场无关;
4. 本站文章均是网友提供,不完全保证技术分享内容的完整性、准确性、时效性、风险性和版权归属;如您发现该文章侵犯了您的权益,可联系我们第一时间进行删除;
5. 本站为非盈利性的个人网站,所有内容不会用来进行牟利,也不会利用任何形式的广告来间接获益,纯粹是为了广大技术爱好者提供技术内容和技术思想的分享性交流网站。

专注分享技术,共同学习,共同进步。侵权联系[81616952@qq.com]

Copyright (C)ICode9.com, All Rights Reserved.

ICode9版权所有