标签:name service app zookeeper value kafka k8s
zookeeper配置istio sidecar后存在的网络不可用问题
如果zookeeper配置了istio sidecar ,在选举阶段就会报connection refused(Connection refused)
错误
这主要是因为 zookeeper 在server之间通信默认是监听 pod IP 地址,而istio要求监听0.0.0.0
,因此需要设置quorumListenOnAllIPs=true
。
具体问题可以参考:https://istio.io/latest/faq/applications/
这个不止在 zookeeper 中会出现,包括 Apache NiFi 、 Cassandra、 Elasticsearch、Redis 中安装 sidecar 模式都会存在这个问题。
由于docker官方的zookeeper镜像没有提供 quorumListenOnAllIPs
的参数,我们需要直接手动添加,详细参考这个issue: https://github.com/31z4/zookeeper-docker/issues/117
或者可以用 bitnami/zookeeper
这个镜像,这个镜像提供了 quorumListenOnAllIPs
支持,可以通过设置ZOO_LISTEN_ALLIPS_ENABLED
环境变量来控制,下面是简单的deployment文件:
Zookeeper集群的安装
创建zookeeper-svc.yaml apiVersion: v1 kind: Service metadata: name: zookeeper-1 labels: app: zookeeper-1 spec: ports: - name: client port: 2181 protocol: TCP - name: follower port: 2888 protocol: TCP - name: leader port: 3888 protocol: TCP selector: app: zookeeper-1 --- apiVersion: v1 kind: Service metadata: name: zookeeper-2 labels: app: zookeeper-2 spec: ports: - name: client port: 2181 protocol: TCP - name: follower port: 2888 protocol: TCP - name: leader port: 3888 protocol: TCP selector: app: zookeeper-2 --- apiVersion: v1 kind: Service metadata: name: zookeeper-3 labels: app: zookeeper-3 spec: ports: - name: client port: 2181 protocol: TCP - name: follower port: 2888 protocol: TCP - name: leader port: 3888 protocol: TCP selector: app: zookeeper-3
创建zookeeper-deployment.yaml kind: Deployment apiVersion: apps/v1 metadata: name: zookeeper-1 spec: replicas: 1 selector: matchLabels: app: zookeeper-1 template: metadata: labels: app: zookeeper-1 spec: containers: - name: zookeeper image: bitnami/zookeeper:3.6.2 imagePullPolicy: Always ports: - containerPort: 2181 env: - name: ALLOW_ANONYMOUS_LOGIN value: "yes" - name: ZOO_LISTEN_ALLIPS_ENABLED value: "true" - name: ZOO_SERVER_ID value: "1" - name: ZOO_SERVERS value: 0.0.0.0:2888:3888,zookeeper-2:2888:3888,zookeeper-3:2888:3888 --- kind: Deployment apiVersion: apps/v1 metadata: name: zookeeper-2 spec: replicas: 1 selector: matchLabels: app: zookeeper-2 template: metadata: labels: app: zookeeper-2 spec: containers: - name: zookeeper image: bitnami/zookeeper:3.6.2 imagePullPolicy: Always ports: - containerPort: 2181 env: - name: ALLOW_ANONYMOUS_LOGIN value: "yes" - name: ZOO_LISTEN_ALLIPS_ENABLED value: "true" - name: ZOO_SERVER_ID value: "2" - name: ZOO_SERVERS value: zookeeper-1:2888:3888,0.0.0.0:2888:3888,zookeeper-3:2888:3888 --- kind: Deployment apiVersion: apps/v1 metadata: name: zookeeper-3 spec: replicas: 1 selector: matchLabels: app: zookeeper-3 template: metadata: labels: app: zookeeper-3 spec: containers: - name: zookeeper image: bitnami/zookeeper:3.6.2 imagePullPolicy: Always ports: - containerPort: 2181 env: - name: ALLOW_ANONYMOUS_LOGIN value: "yes" - name: ZOO_LISTEN_ALLIPS_ENABLED value: "true" - name: ZOO_SERVER_ID value: "3" - name: ZOO_SERVERS value: zookeeper-1:2888:3888,zookeeper-2:2888:3888,0.0.0.0:2888:3888
执行
kubectl apply -f zookeeper-svc.yaml -n zookeeper
kubectl apply -f zookeeper-deployment.yaml -n zookeeper
Kafka集群的安装
创建kafka-svc.yaml apiVersion: v1 kind: Service metadata: name: kafka-service-1 labels: app: kafka-service-1 spec: type: NodePort ports: - port: 9092 name: kafka-service-1 targetPort: 9092 nodePort: 30901 protocol: TCP selector: app: kafka-service-1 --- apiVersion: v1 kind: Service metadata: name: kafka-service-2 labels: app: kafka-service-2 spec: type: NodePort ports: - port: 9092 name: kafka-service-2 targetPort: 9092 nodePort: 30902 protocol: TCP selector: app: kafka-service-2 --- apiVersion: v1 kind: Service metadata: name: kafka-service-3 labels: app: kafka-service-3 spec: type: NodePort ports: - port: 9092 name: kafka-service-3 targetPort: 9092 nodePort: 30903 protocol: TCP selector: app: kafka-service-3
创建kafka-deployment.yaml(注意修改我尖括号的地方)
查看CLUSTER-IP可以使用kubectl get svc -n zookeeper,注意KAFKA_ADVERTISED_LISTENERS变量,我就是在这踩的坑,发现这里不写的话命令行能用,但使用java程序去连接会因为没有代理而直接连接内网地址,最终导致连接不上node而出错(你会发现程序会去直接连接CLUSTER-IP)
apiVersion: apps/v1 kind: Deployment metadata: name: kafka-deployment-1 spec: replicas: 1 selector: matchLabels: name: kafka-service-1 template: metadata: labels: name: kafka-service-1 app: kafka-service-1 spec: containers: - name: kafka-1 image: wurstmeister/kafka imagePullPolicy: IfNotPresent ports: - containerPort: 9092 env: - name: KAFKA_ADVERTISED_PORT value: "9092" - name: KAFKA_ADVERTISED_HOST_NAME value: <kafka-svc1-CLUSTER-IP> - name: KAFKA_ZOOKEEPER_CONNECT value: zoo1:2181,zoo2:2181,zoo3:2181 - name: KAFKA_BROKER_ID value: "1" - name: KAFKA_CREATE_TOPICS value: mytopic:2:1 - name: KAFKA_ADVERTISED_LISTENERS value: PLAINTEXT://<master-ip例如192.168.128.52>:30901 - name: KAFKA_LISTENERS value: PLAINTEXT://0.0.0.0:9092 --- apiVersion: apps/v1 kind: Deployment metadata: name: kafka-deployment-2 spec: replicas: 1 selector: matchLabels: name: kafka-service-2 template: metadata: labels: name: kafka-service-2 app: kafka-service-2 spec: containers: - name: kafka-2 image: wurstmeister/kafka imagePullPolicy: IfNotPresent ports: - containerPort: 9092 env: - name: KAFKA_ADVERTISED_PORT value: "9092" - name: KAFKA_ADVERTISED_HOST_NAME value: <kafka-svc2-CLUSTER-IP> - name: KAFKA_ZOOKEEPER_CONNECT value: zoo1:2181,zoo2:2181,zoo3:2181 - name: KAFKA_BROKER_ID value: "2" - name: KAFKA_ADVERTISED_LISTENERS value: PLAINTEXT://<master-ip例如192.168.128.52>:30902 - name: KAFKA_LISTENERS value: PLAINTEXT://0.0.0.0:9092 --- apiVersion: apps/v1 kind: Deployment metadata: name: kafka-deployment-3 spec: replicas: 1 selector: matchLabels: name: kafka-service-3 template: metadata: labels: name: kafka-service-3 app: kafka-service-3 spec: containers: - name: kafka-3 image: wurstmeister/kafka imagePullPolicy: IfNotPresent ports: - containerPort: 9092 env: - name: KAFKA_ADVERTISED_PORT value: "9092" - name: KAFKA_ADVERTISED_HOST_NAME value: <kafka-svc3-CLUSTER-IP> - name: KAFKA_ZOOKEEPER_CONNECT value: zoo1:2181,zoo2:2181,zoo3:2181 - name: KAFKA_BROKER_ID value: "3" - name: KAFKA_ADVERTISED_LISTENERS value: PLAINTEXT://<master-ip例如192.168.128.52>:30903 - name: KAFKA_LISTENERS value: PLAINTEXT://0.0.0.0:9092
测试
命令行
你可以进入任意一个pod然后使用命令行进行kafka的操作,常用的命令和目录我放下面了
kubectl exec -it kafka-deployment-1-xxxxxxxxxxx -n zookeeper /bin/bash
cd cd opt/kafka
# 查看topics
bin/kafka-topics.sh --list --zookeeper <任意zookeeper-svc-clusterIP>:2181
# 手动创建主题
bin/kafka-topics.sh --create --zookeeper <zookeeper-svc1-clusterIP>:2181,<zookeeper-svc2-clusterIP>:2181,<zookeeper-svc3-clusterIP>:2181 --topic test --partitions 3 --replication-factor 1
# 写(CTRL+D结束写内容)
bin/kafka-console-producer.sh --broker-list <kafka-svc1-clusterIP>:9092,<kafka-svc2-clusterIP>:9092,<kafka-svc3-clusterIP>:9092 --topic test
# 读(CTRL+C结束读内容)
bin/kafka-console-consumer.sh --bootstrap-server <任意kafka-svc-clusterIP>:9092 --topic test --from-beginning
标签:name,service,app,zookeeper,value,kafka,k8s 来源: https://www.cnblogs.com/banger/p/16292734.html
本站声明: 1. iCode9 技术分享网(下文简称本站)提供的所有内容,仅供技术学习、探讨和分享; 2. 关于本站的所有留言、评论、转载及引用,纯属内容发起人的个人观点,与本站观点和立场无关; 3. 关于本站的所有言论和文字,纯属内容发起人的个人观点,与本站观点和立场无关; 4. 本站文章均是网友提供,不完全保证技术分享内容的完整性、准确性、时效性、风险性和版权归属;如您发现该文章侵犯了您的权益,可联系我们第一时间进行删除; 5. 本站为非盈利性的个人网站,所有内容不会用来进行牟利,也不会利用任何形式的广告来间接获益,纯粹是为了广大技术爱好者提供技术内容和技术思想的分享性交流网站。