ICode9

精准搜索请尝试: 精确搜索
首页 > 其他分享> 文章详细

大数据-zookeeper的数据结构及节点操作

2019-11-14 14:00:11  阅读:233  来源: 互联网

标签:13 zk zookeeper 2181 CONNECTED 2018 数据结构 节点 localhost


1         zookeeper结构和命令

1.1       zookeeper两大功能:

读写数据:往往是状态信息或配置信息

提供监听:发现数据的变化情况

 

 

1.2       zookeeper特性

1、Zookeeper:一个leader,多个follower组成的集群

2、 全局数据一致:每个server保存一份相同的数据副本,client无论连接到哪个server,数据都是一致的

3、分布式读写,更新请求转发,由leader实施

4、更新请求顺序进行,来自同一个client的更新请求按其发送顺序依次执行

5、数据更新原子性,一次数据更新要么成功,要么失败

6、实时性,在一定时间范围内,client能读到最新数据

 

 

1.3       zookeeper数据结构

1、层次化的目录结构,命名符合常规文件系统规范(见下图)

2、每个节点在zookeeper中叫做znode,并且其有一个唯一的路径标识

3、节点Znode可以包含数据和子节点(但是EPHEMERAL类型的节点不能有子节点,下一页详细讲解)

4、客户端应用可以在节点上设置监视器(后续详细讲解)

 

 

1.4       数据结构图

wps1A0D

 

 

1.5       读写节点数据:

1.5.1     命令行连接zk服务器

zkCli.sh

返回如下信息:

[root@min1 ~]# zkCli.sh

Connecting to localhost:2181

2018-11-13 07:30:14,084 [myid:] - INFO  [main:Environment@100] - Client environment:zookeeper.version=3.4.5-1392090, built on 09/30/2012 17:52 GMT

2018-11-13 07:30:14,094 [myid:] - INFO  [main:Environment@100] - Client environment:host.name=min1

2018-11-13 07:30:14,095 [myid:] - INFO  [main:Environment@100] - Client environment:java.version=1.7.0_45

2018-11-13 07:30:14,099 [myid:] - INFO  [main:Environment@100] - Client environment:java.vendor=Oracle Corporation

2018-11-13 07:30:14,099 [myid:] - INFO  [main:Environment@100] - Client environment:java.home=/root/apps/jdk1.7.0_45/jre

2018-11-13 07:30:14,100 [myid:] - INFO  [main:Environment@100] - Client environment:java.class.path=/root/apps/zookeeper/bin/../build/classes:/root/apps/zookeeper/bin/../build/lib/*.jar:/root/apps/zookeeper/bin/../lib/slf4j-log4j12-1.6.1.jar:/root/apps/zookeeper/bin/../lib/slf4j-api-1.6.1.jar:/root/apps/zookeeper/bin/../lib/netty-3.2.2.Final.jar:/root/apps/zookeeper/bin/../lib/log4j-1.2.15.jar:/root/apps/zookeeper/bin/../lib/jline-0.9.94.jar:/root/apps/zookeeper/bin/../zookeeper-3.4.5.jar:/root/apps/zookeeper/bin/../src/java/lib/*.jar:/root/apps/zookeeper/bin/../conf:

2018-11-13 07:30:14,100 [myid:] - INFO  [main:Environment@100] - Client environment:java.library.path=/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib

2018-11-13 07:30:14,101 [myid:] - INFO  [main:Environment@100] - Client environment:java.io.tmpdir=/tmp

2018-11-13 07:30:14,101 [myid:] - INFO  [main:Environment@100] - Client environment:java.compiler=<NA>

2018-11-13 07:30:14,102 [myid:] - INFO  [main:Environment@100] - Client environment:os.name=Linux

2018-11-13 07:30:14,103 [myid:] - INFO  [main:Environment@100] - Client environment:os.arch=amd64

2018-11-13 07:30:14,103 [myid:] - INFO  [main:Environment@100] - Client environment:os.version=2.6.32-573.el6.x86_64

2018-11-13 07:30:14,104 [myid:] - INFO  [main:Environment@100] - Client environment:user.name=root

2018-11-13 07:30:14,105 [myid:] - INFO  [main:Environment@100] - Client environment:user.home=/root

2018-11-13 07:30:14,106 [myid:] - INFO  [main:Environment@100] - Client environment:user.dir=/root

2018-11-13 07:30:14,108 [myid:] - INFO  [main:ZooKeeper@438] - Initiating client connection, connectString=localhost:2181 sessionTimeout=30000 watcher=org.apache.zookeeper.ZooKeeperMain$MyWatcher@5cf2baac

Welcome to ZooKeeper!

2018-11-13 07:30:14,134 [myid:] - INFO  [main-SendThread(localhost:2181):ClientCnxn$SendThread@966] - Opening socket connection to server localhost/127.0.0.1:2181. Will not attempt to authenticate using SASL (unknown error)

2018-11-13 07:30:14,149 [myid:] - INFO  [main-SendThread(localhost:2181):ClientCnxn$SendThread@849] - Socket connection established to localhost/127.0.0.1:2181, initiating session

JLine support is enabled

2018-11-13 07:30:14,179 [myid:] - INFO  [main-SendThread(localhost:2181):ClientCnxn$SendThread@1207] - Session establishment complete on server localhost/127.0.0.1:2181, sessionid = 0x1670a2f4010000a, negotiated timeout = 30000

 

WATCHER::

 

WatchedEvent state:SyncConnected type:None path:null

[zk: localhost:2181(CONNECTED) 0]

 

1.5.2     切换连接的服务器

connect 192.168.33.62:2181  connect min2:2181

返回:

[zk: localhost:2181(CONNECTED) 1] connect min2:2181

2018-11-13 07:31:44,978 [myid:] - INFO  [main-EventThread:ClientCnxn$EventThread@509] - EventThread shut down

2018-11-13 07:31:44,979 [myid:] - INFO  [main:ZooKeeper@684] - Session: 0x1670a2f4010000a closed

2018-11-13 07:31:44,985 [myid:] - INFO  [main:ZooKeeper@438] - Initiating client connection, connectString=min2:2181 sessionTimeout=30000 watcher=org.apache.zookeeper.ZooKeeperMain$MyWatcher@7bae9cec

2018-11-13 07:31:44,988 [myid:] - INFO  [main-SendThread(min2:2181):ClientCnxn$SendThread@966] - Opening socket connection to server min2/192.168.33.62:2181. Will not attempt to authenticate using SASL (unknown error)

ZooKeeper -server host:port cmd args

        connect host:port

        get path [watch]

        ls path [watch]

        set path data [version]

        rmr path

        delquota [-n|-b] path

        quit

        printwatches on|off

        create [-s] [-e] path data acl

        stat path [watch]

        close

        ls2 path [watch]

        history

        listquota path

        setAcl path acl

        getAcl path

        sync path

        redo cmdno

        addauth scheme auth

        delete path [version]

        setquota -n|-b val path

[zk: min2:2181(CONNECTING) 2] 2018-11-13 07:31:44,990 [myid:] - INFO  [main-SendThread(min2:2181):ClientCnxn$SendThread@849] - Socket connection established to min2/192.168.33.62:2181, initiating session

2018-11-13 07:31:44,996 [myid:] - INFO  [main-SendThread(min2:2181):ClientCnxn$SendThread@1207] - Session establishment complete on server min2/192.168.33.62:2181, sessionid = 0x2670a2f1b430005, negotiated timeout = 30000

 

WATCHER::

 

WatchedEvent state:SyncConnected type:None path:null

 

[zk: min2:2181(CONNECTED) 2]

 

1.5.3     查看节点

ls

[zk: min2:2181(CONNECTED) 7] ls /

[zookeeper]

 

1.5.4     创建节点

create /app1 "this is app1 servers parent"

[zk: min2:2181(CONNECTED) 8] create /app1 "this is app1 servers parent"

Created /app1

ls

[zk: min2:2181(CONNECTED) 10] ls /

[app1, zookeeper]

 create /app1/server01 "192.168.33.61,100"

[zk: min2:2181(CONNECTED) 11] create /app1/server01 "192.168.33.61,100"

Created /app1/server01

 

[zk: min2:2181(CONNECTED) 12] ls /app1

[server01]

1.5.5     获取节点数据

get /app1/server01

[zk: localhost:2181(CONNECTED) 0] get /app1/server01

"192.168.33.61,100"

cZxid = 0x200000023  #内部事务编号

ctime = Tue Nov 13 07:58:10 CST 2018 #创建时间

mZxid = 0x200000023

mtime = Tue Nov 13 07:58:10 CST 2018 #修改时间

pZxid = 0x200000023 #持久化事务

cversion = 0 #创建版本

dataVersion = 0 #数据版本

aclVersion = 0

ephemeralOwner = 0x0

dataLength = 19 #数据长度

numChildren = 0 #子节点数

1.5.6     创建一个短暂节点

1.5.6.1      创建短暂节点

[zk: localhost:2181(CONNECTED) 2] create -e /app-emphemeral 181818

Created /app-emphemeral

1.5.6.2      查看节点信息

[zk: localhost:2181(CONNECTED) 4] ls /

[app1, app-emphemeral, zookeeper] #可以看到临时节点

1.5.6.3      退出后再链接进去查看节点信息

quit

zkCli.sh

ls /

[zk: localhost:2181(CONNECTED) 0] ls /

[app1, zookeeper]   #临时节点已经没有了;

1.5.7     创建带有序号的节点

[zk: localhost:2181(CONNECTED) 18] create /test 888

Created /test

[zk: localhost:2181(CONNECTED) 19] create -s /test/aa 999

Created /test/aa0000000000

继续创建

[zk: localhost:2181(CONNECTED) 20] create -s /test/aa 999

Created /test/aa0000000001   #会自动增加序号添加节点

[zk: localhost:2181(CONNECTED) 21] create -s /test/aa 999

Created /test/aa0000000002

[zk: localhost:2181(CONNECTED) 22] create -s /test/aa 999

Created /test/aa0000000003

[zk: localhost:2181(CONNECTED) 23] create -s /test/aa 999

Created /test/aa0000000004

[zk: localhost:2181(CONNECTED) 24] create -s /test/aa 999

Created /test/aa0000000005

[zk: localhost:2181(CONNECTED) 25] create -s /test/aa 999

Created /test/aa0000000006

[zk: localhost:2181(CONNECTED) 26] create -s /test/aa 999

Created /test/aa0000000007

[zk: localhost:2181(CONNECTED) 27] create -s /test/bb 999

Created /test/bb0000000008

[zk: localhost:2181(CONNECTED) 28] create -s /test/bb 999

Created /test/bb0000000009

[zk: localhost:2181(CONNECTED) 29] create -s /test/cc 999

Created /test/cc0000000010

[zk: localhost:2181(CONNECTED) 30] create -s /test/cc 999

创建临时的带序号的节点

[zk: localhost:2181(CONNECTED) 31] create -e -s /test/cc 999

Created /test/cc0000000012

1.5.8     多服务器更新数据同步

61上操作:

[zk: localhost:2181(CONNECTED) 33] set /app1 111

cZxid = 0x200000022

ctime = Tue Nov 13 07:55:17 CST 2018

mZxid = 0x200000041

mtime = Tue Nov 13 08:41:24 CST 2018

pZxid = 0x200000023

cversion = 1

dataVersion = 1

aclVersion = 0

ephemeralOwner = 0x0

dataLength = 3

numChildren = 1

[zk: localhost:2181(CONNECTED) 34] get /app1

111

cZxid = 0x200000022

ctime = Tue Nov 13 07:55:17 CST 2018

mZxid = 0x200000041

mtime = Tue Nov 13 08:41:24 CST 2018

pZxid = 0x200000023

cversion = 1

dataVersion = 1

aclVersion = 0

ephemeralOwner = 0x0

dataLength = 3

numChildren = 1

[zk: localhost:2181(CONNECTED) 35]

登录62,操作:

zkCli.sh  #登录62本机

[zk: localhost:2181(CONNECTED) 0] get /app1

111

cZxid = 0x200000022

ctime = Tue Nov 13 07:55:17 CST 2018

mZxid = 0x200000041

mtime = Tue Nov 13 08:41:24 CST 2018

pZxid = 0x200000023

cversion = 1

dataVersion = 1

aclVersion = 0

ephemeralOwner = 0x0

dataLength = 3

numChildren = 1

[zk: localhost:2181(CONNECTED) 1] set /app1 222

cZxid = 0x200000022

ctime = Tue Nov 13 07:55:17 CST 2018

mZxid = 0x200000046

mtime = Tue Nov 13 08:43:54 CST 2018

pZxid = 0x200000023

cversion = 1

dataVersion = 2

aclVersion = 0

ephemeralOwner = 0x0

dataLength = 3

numChildren = 1

[zk: localhost:2181(CONNECTED) 2] get /app1

222

cZxid = 0x200000022

ctime = Tue Nov 13 07:55:17 CST 2018

mZxid = 0x200000046

mtime = Tue Nov 13 08:43:54 CST 2018

pZxid = 0x200000023

cversion = 1

dataVersion = 2

aclVersion = 0

ephemeralOwner = 0x0

dataLength = 3

numChildren = 1

 

在61上查看

[zk: localhost:2181(CONNECTED) 0] get /app1

222

cZxid = 0x200000022

ctime = Tue Nov 13 07:55:17 CST 2018

mZxid = 0x200000046

mtime = Tue Nov 13 08:43:54 CST 2018

pZxid = 0x200000023

cversion = 1

dataVersion = 2

aclVersion = 0

ephemeralOwner = 0x0

dataLength = 3

numChildren = 1

 

 

 

1.6       监听事件

 

1.6.1     数据变化监听

61上监听:

[zk: localhost:2181(CONNECTED) 1] get /app1 watch

222

cZxid = 0x200000022

ctime = Tue Nov 13 07:55:17 CST 2018

mZxid = 0x200000046

mtime = Tue Nov 13 08:43:54 CST 2018

pZxid = 0x200000023

cversion = 1

dataVersion = 2

aclVersion = 0

ephemeralOwner = 0x0

dataLength = 3

numChildren = 1

[zk: localhost:2181(CONNECTED) 2]

WATCHER::

 

62上修改

[zk: localhost:2181(CONNECTED) 3] set /app1 333

cZxid = 0x200000022

ctime = Tue Nov 13 07:55:17 CST 2018

mZxid = 0x200000048

mtime = Tue Nov 13 08:55:45 CST 2018

pZxid = 0x200000023

cversion = 1

dataVersion = 3

aclVersion = 0

ephemeralOwner = 0x0

dataLength = 3

numChildren = 1

 

在61上发生变化:注意,该事件只接受一次;再修改就不会接受到监听了;

[zk: localhost:2181(CONNECTED) 2]

WATCHER::

WatchedEvent state:SyncConnected type:NodeDataChanged path:/app1

 

1.6.2     子节点变化监听

61上开启节点监听

[zk: localhost:2181(CONNECTED) 4] ls /app1 watch

[server01]

62上操作,增加节点

[zk: localhost:2181(CONNECTED) 4] create /app1/app2 222

Created /app1/app2

查看61

[zk: localhost:2181(CONNECTED) 5]

WATCHER::

 

WatchedEvent state:SyncConnected type:NodeChildrenChangedpath:/app1

 

[zk: localhost:2181(CONNECTED) 5] ls /app1

[app2, server01]

 

1.7       节点类型

1.7.1     Znode有两种类型:

短暂(ephemeral)(断开连接自己删除)

    如果创建的客户端断开,那么会被自动删除;

     用于服务器掉线的自动感知,掉线后自动删除,那么就知道不存在了;

持久(persistent)(断开连接不删除)

如果创建的客户端断开,那么不会被自动删除;

 

1.7.2     Znode有四种形式的目录节点(默认是persistent )

PERSISTENT(持久节点)

PERSISTENT_SEQUENTIAL(持久序列/test0000000019 )

EPHEMERAL(临时节点)

EPHEMERAL_SEQUENTIAL (临时序列/test0000000019)

创建znode时设置顺序标识,znode名称后会附加一个值,顺序号是一个单调递增的计数器,由父节点维护

在分布式系统中,顺序号可以被用于为所有的事件进行全局排序,这样客户端可以通过顺序号推断事件的顺序

标签:13,zk,zookeeper,2181,CONNECTED,2018,数据结构,节点,localhost
来源: https://www.cnblogs.com/qq18361642/p/11856635.html

本站声明: 1. iCode9 技术分享网(下文简称本站)提供的所有内容,仅供技术学习、探讨和分享;
2. 关于本站的所有留言、评论、转载及引用,纯属内容发起人的个人观点,与本站观点和立场无关;
3. 关于本站的所有言论和文字,纯属内容发起人的个人观点,与本站观点和立场无关;
4. 本站文章均是网友提供,不完全保证技术分享内容的完整性、准确性、时效性、风险性和版权归属;如您发现该文章侵犯了您的权益,可联系我们第一时间进行删除;
5. 本站为非盈利性的个人网站,所有内容不会用来进行牟利,也不会利用任何形式的广告来间接获益,纯粹是为了广大技术爱好者提供技术内容和技术思想的分享性交流网站。

专注分享技术,共同学习,共同进步。侵权联系[81616952@qq.com]

Copyright (C)ICode9.com, All Rights Reserved.

ICode9版权所有