标签：haproxy master1 流程 192.168 kubeconfig sefe kube root 163
文章目录
master高可用部署流程
- 去这篇博客，内容过多，需要6台虚拟机，有条件的建议跟着实验完整做一遍，一遍更深层次的理解高可用：
【kubernetes】k8s集群高可用部署安装和概念详细说明【含离线部署】，客户端连接haproxy访问高可用流程
客户端连接haproxy访问高可用集群
环境确认与准备【必看】
- 客户端连接haproxy访问高可用集群，这句话直观吗？
是这个意思，就是我们使用任意集群外的主机，使用kubeconfig的形式连接到haproxy，然后haproxy会自动转发到master，所以只需要连接haproxy，就可以实现集群高可用了【2个master死其中一个无所谓的】 - 首先确保上面文章中高可用部署完整且测试正常，然后6台虚拟机都需要开机
[root@master1-163 ~]# kubectl get nodes
NAME STATUS ROLES AGE VERSION
master1-163 Ready control-plane,master 22h v1.21.1
master2-162 Ready control-plane,master 21h v1.21.1
worker-165 Ready <none> 17h v1.21.1
[root@master1-163 ~]#
[root@master1-163 ~]# kubectl get pods -A -owide
NAMESPACE NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
kube-system calico-kube-controllers-78d6f96c7b-nwbmt 1/1 Running 0 16h 10.244.139.67 worker-165 <none> <none>
kube-system calico-node-nh977 1/1 Running 0 16h 192.168.59.162 master2-162 <none> <none>
kube-system calico-node-s89tx 1/1 Running 1 16h 192.168.59.163 master1-163 <none> <none>
kube-system calico-node-vt5dn 1/1 Running 0 16h 192.168.59.165 worker-165 <none> <none>
kube-system coredns-545d6fc579-6l9xs 1/1 Running 0 22h 10.244.139.66 worker-165 <none> <none>
kube-system coredns-545d6fc579-mrm2w 1/1 Running 0 22h 10.244.139.65 worker-165 <none> <none>
kube-system kube-apiserver-master1-163 1/1 Running 2 22h 192.168.59.163 master1-163 <none> <none>
kube-system kube-apiserver-master2-162 1/1 Running 1 21h 192.168.59.162 master2-162 <none> <none>
kube-system kube-controller-manager-master1-163 1/1 Running 2 22h 192.168.59.163 master1-163 <none> <none>
kube-system kube-controller-manager-master2-162 1/1 Running 1 21h 192.168.59.162 master2-162 <none> <none>
kube-system kube-proxy-kp8p6 1/1 Running 1 21h 192.168.59.162 master2-162 <none> <none>
kube-system kube-proxy-kqg72 1/1 Running 2 22h 192.168.59.163 master1-163 <none> <none>
kube-system kube-proxy-nftgv 1/1 Running 0 17h 192.168.59.165 worker-165 <none> <none>
kube-system kube-scheduler-master1-163 1/1 Running 2 22h 192.168.59.163 master1-163 <none> <none>
kube-system kube-scheduler-master2-162 1/1 Running 1 21h 192.168.59.162 master2-162 <none> <none>
[root@master1-163 ~]#
- 最后，找一台客户端用来访问haproxy，找一台不属于集群一部分的机器。
我这就用之前的etcd1来做客户端主机吧
Last login: Fri Nov 26 12:58:16 2021 from 192.168.59.1
[root@etcd1 ~]#
[root@etcd1 ~]# ip a | grep 59
inet 192.168.59.156/24 brd 192.168.59.255 scope global ens32
[root@etcd1 ~]#
客户端连接happroxy说明
-
client连接happroxy有2种方式
- 1、kubeconfig的方式
- 2、token的方式
-
我之前文章中对kubeconfig和token都有说明，想了解的看这篇
【Kubernetes】k8s的安全管理详细说明【k8s框架说明、token验证和kubeconfig验证详细说明】 -
我这使用kubeconfig的方式来访问
kubeconfig配置【master上操作】
-
我也是跟着上面文章中配置的，我这不对命令做说明，不清楚的自行去看上面那篇文章中的说明哈
-
在任意一个master上操作即可，我在master1上操作吧
下面连接地址改为happroxy的地址，已经回车一个空行，并做注释了，注意看
[root@master1-163 ~]# mkdir sefe
[root@master1-163 ~]# cd sefe
[root@master1-163 sefe]# openssl genrsa -out ccx.key 2048
Generating RSA private key, 2048 bit long modulus
..........................+++
....+++
e is 65537 (0x10001)
[root@master1-163 sefe]# openssl req -new -key ccx.key -out ccx.csr -subj "/CN=ccx/O=cka2021"
[root@master1-163 sefe]# cat ccx.csr | base64 | tr -d "\n"
LS0tLS1CRUdJTiBDRVJUSUZJQ0FURSBSRVFVRVNULS0tLS0KTUlJQ1pUQ0NBVTBDQVFBd0lERU1NQW9HQTFVRUF3d0RZMk40TVJBd0RnWURWUVFLREFkamEyRXlNREl4TUlJQgpJakFOQmdrcWhraUc5dzBCQVFFRkFBT0NBUThBTUlJQkNnS0NBUUVBMjRMbDlPWXR6RE9oYlRZN0o5aTJ4VmpJCnFBdmZhS3BaOFJxaE5uSmxnRFMvWUlTVFhwT3k2T05KTDZ5UWY3akMxdk9OZ2tsRkZmdkt6azE2N3RRdityWGQKK1VYSkZ3a2RQWkttZlZseXhNWkRXQlhRSElvVFNvWjhTcW0va25BUDRCNURpWVkwR3QxSW52MkV6WkxrSDQxSwo1ajJCMHB4ekFDNUhRclMwc1k5RkgxY3JqdmhHOGsvOHFxKzRUUnB2K01BeG5LdE5RWFhyOGUvTzV6S3R2Um53CjA0aGFMMEV6VGlHODhQOWIzUEpSTTZXbjhNbVVrclZFTnUrSFJKTkYyOFVtRGlUUDFEeWZ0SHZUWXR6WlJqMDIKOUUrS1I3WUVkK05uU25uTnR6WFNIRTBXMUFtdDNKdFloeVNyM29idUZ6NllIdy9mcEs5SXJzaWZsaGtqa3dJRApBUUFCb0FBd0RRWUpLb1pJaHZjTkFRRUxCUUFEZ2dFQkFBWGp6d2loaTRVYXR2cUhTZXNUaEs4R2phclBsNHJ5ClFFSGhvMCtZdzVnelNVMXVtRXN3Tm9TVmdHc25BZmltT1lwcURUTWpmbzN5S2VrNjM1SDlpV0s3MDdMNEtHRlgKVENTRUh3ZFNrbys4a1B2dTB5VWVGd2pnSSsxdHF3K0puRm5maGg5bjVaSmhyaFM0Z2dJTythS2dBeDA0REg0SgpoVVQzdkZDZEVhM1pLcEZUQ1A0Ti9TWXdQVWZmZDQ0QW4zWHBQN0RLUGVOM3dJUFdXNmt1eWM5aFlzUWc4SVd6CmFieU9GQzFsS2lFUnpORzc5S2x1aCs0SWFIRE91di9OZ3ZMTUliSzd5TmdHWlJrVzRSNENXb3lEdEgxZ2o1SG4KL0NqOWczNWdQMFlFaTZ6U05TenFhdXJoUm5pWTFtQ2xVTGMyVXN0Ky8zV3hGNTZ6NlRVbWhuQT0KLS0tLS1FTkQgQ0VSVElGSUNBVEUgUkVRVUVTVC0tLS0tCg==[root@master1-163 sefe]#
[root@master1-163 sefe]# cat csr.yaml
apiVersion: certificates.k8s.io/v1beta1
kind: CertificateSigningRequest
metadata:
name: ccx
spec:
groups:
- system:authenticated
request: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURSBSRVFVRVNULS0tLS0KTUlJQ1pUQ0NBVTBDQVFBd0lERU1NQW9HQTFVRUF3d0RZMk40TVJBd0RnWURWUVFLREFkamEyRXlNREl4TUlJQgpJakFOQmdrcWhraUc5dzBCQVFFRkFBT0NBUThBTUlJQkNnS0NBUUVBMjRMbDlPWXR6RE9oYlRZN0o5aTJ4VmpJCnFBdmZhS3BaOFJxaE5uSmxnRFMvWUlTVFhwT3k2T05KTDZ5UWY3akMxdk9OZ2tsRkZmdkt6azE2N3RRdityWGQKK1VYSkZ3a2RQWkttZlZseXhNWkRXQlhRSElvVFNvWjhTcW0va25BUDRCNURpWVkwR3QxSW52MkV6WkxrSDQxSwo1ajJCMHB4ekFDNUhRclMwc1k5RkgxY3JqdmhHOGsvOHFxKzRUUnB2K01BeG5LdE5RWFhyOGUvTzV6S3R2Um53CjA0aGFMMEV6VGlHODhQOWIzUEpSTTZXbjhNbVVrclZFTnUrSFJKTkYyOFVtRGlUUDFEeWZ0SHZUWXR6WlJqMDIKOUUrS1I3WUVkK05uU25uTnR6WFNIRTBXMUFtdDNKdFloeVNyM29idUZ6NllIdy9mcEs5SXJzaWZsaGtqa3dJRApBUUFCb0FBd0RRWUpLb1pJaHZjTkFRRUxCUUFEZ2dFQkFBWGp6d2loaTRVYXR2cUhTZXNUaEs4R2phclBsNHJ5ClFFSGhvMCtZdzVnelNVMXVtRXN3Tm9TVmdHc25BZmltT1lwcURUTWpmbzN5S2VrNjM1SDlpV0s3MDdMNEtHRlgKVENTRUh3ZFNrbys4a1B2dTB5VWVGd2pnSSsxdHF3K0puRm5maGg5bjVaSmhyaFM0Z2dJTythS2dBeDA0REg0SgpoVVQzdkZDZEVhM1pLcEZUQ1A0Ti9TWXdQVWZmZDQ0QW4zWHBQN0RLUGVOM3dJUFdXNmt1eWM5aFlzUWc4SVd6CmFieU9GQzFsS2lFUnpORzc5S2x1aCs0SWFIRE91di9OZ3ZMTUliSzd5TmdHWlJrVzRSNENXb3lEdEgxZ2o1SG4KL0NqOWczNWdQMFlFaTZ6U05TenFhdXJoUm5pWTFtQ2xVTGMyVXN0Ky8zV3hGNTZ6NlRVbWhuQT0KLS0tLS1FTkQgQ0VSVElGSUNBVEUgUkVRVUVTVC0tLS0tCg==
usages:
- client auth
[root@master1-163 sefe]#
[root@master1-163 sefe]# kubectl apply -f csr.yaml
Warning: certificates.k8s.io/v1beta1 CertificateSigningRequest is deprecated in v1.19+, unavailable in v1.22+; use certificates.k8s.io/v1 CertificateSigningRequest
certificatesigningrequest.certificates.k8s.io/ccx created
[root@master1-163 sefe]#
[root@master1-163 sefe]# kubectl get csr
NAME AGE SIGNERNAME REQUESTOR CONDITION
ccx 23s kubernetes.io/legacy-unknown kubernetes-admin Pending
[root@master1-163 sefe]#
[root@master1-163 sefe]#
[root@master1-163 sefe]# kubectl certificate approve ccx
certificatesigningrequest.certificates.k8s.io/ccx approved
[root@master1-163 sefe]# kubectl get csr
NAME AGE SIGNERNAME REQUESTOR CONDITION
ccx 2m32s kubernetes.io/legacy-unknown kubernetes-admin Approved,Issued
[root@master1-163 sefe]# kubectl get csr/ccx -o jsonpath='{.status.certificate}' | base64 -d > ccx.crt
[root@master1-163 sefe]#
[root@master1-163 sefe]# cat ccx.crt
-----BEGIN CERTIFICATE-----
MIIDBjCCAe6gAwIBAgIRAJntISUbREJeqXDV7z2+HQUwDQYJKoZIhvcNAQELBQAw
FTETMBEGA1UEAxMKa3ViZXJuZXRlczAeFw0yMTExMzAwMjUwMjdaFw0yMjExMzAw
MjUwMjdaMCAxEDAOBgNVBAoTB2NrYTIwMjExDDAKBgNVBAMTA2NjeDCCASIwDQYJ
KoZIhvcNAQEBBQADggEPADCCAQoCggEBANuC5fTmLcwzoW02OyfYtsVYyKgL32iq
WfEaoTZyZYA0v2CEk16TsujjSS+skH+4wtbzjYJJRRX7ys5Neu7UL/q13flFyRcJ
HT2Spn1ZcsTGQ1gV0ByKE0qGfEqpv5JwD+AeQ4mGNBrdSJ79hM2S5B+NSuY9gdKc
cwAuR0K0tLGPRR9XK474RvJP/KqvuE0ab/jAMZyrTUF16/Hvzucyrb0Z8NOIWi9B
M04hvPD/W9zyUTOlp/DJlJK1RDbvh0STRdvFJg4kz9Q8n7R702Lc2UY9NvRPike2
BHfjZ0p5zbc10hxNFtQJrdybWIckq96G7hc+mB8P36SvSK7In5YZI5MCAwEAAaNG
MEQwEwYDVR0lBAwwCgYIKwYBBQUHAwIwDAYDVR0TAQH/BAIwADAfBgNVHSMEGDAW
gBRFBBI4o0uRg0+Dv6LzwLmyou1PCDANBgkqhkiG9w0BAQsFAAOCAQEAJNE1NuAs
o5cnRU+65ys/+xxBt3Fg8DMj0HokLumvFtZ7CDahX5fPHt6YynAfulgdbhba01UV
c854sOfDO8xYEuaRLCKn+a+yYH7QCOztlJGHlaKiYk7JERdj0u199gpJ+ANLoSQP
fJVwwfclioFf16UDPXNocSJQrjWih34HNrudCy1XjPOu7etjT2ICQ1LD04w77Ls/
speOYBrOQFR33Utn0s/xoGI8ExPzCSpT1Zc/JwRDUoaD6Lu83XaMD24ip+Jj13TY
uI62+u0VfMvp8eiS6MigwP7w7vMD6XqyDF1yXmsnAFspEhuiJcG6fGO2OnkcRb8P
Q5c+ijY0QDP3GA==
-----END CERTIFICATE-----
[root@master1-163 sefe]#
[root@master1-163 sefe]# kubectl create clusterrolebinding test1 --clusterrole=cluster-admin --user=ccx
clusterrolebinding.rbac.authorization.k8s.io/test1 created
[root@master1-163 sefe]# cp /etc/kubernetes/pki/ca.crt .
[root@master1-163 sefe]#
# 下面地址改为：happroxy的ip
[root@master1-163 sefe]# kubectl config --kubeconfig=kc1 set-cluster cluster1 --server=https://192.168.59.164:6443 --certificate-authority=ca.crt --embed-certs=true
Cluster "cluster1" set.
[root@master1-163 sefe]# kubectl config --kubeconfig=kc1 set-credentials ccx --client-certificate=ccx.crt --client-key=ccx.key --embed-certs=true
User "ccx" set.
[root@master1-163 sefe]#
[root@master1-163 sefe]# kubectl config --kubeconfig=kc1 set-context context1 --cluster=cluster1 --namespace=default --user=ccx
Context "context1" created.
[root@master1-163 sefe]# vi kc1
[root@master1-163 sefe]# cat kc1
apiVersion: v1
clusters:
- cluster:
certificate-authority-data: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUM1ekNDQWMrZ0F3SUJBZ0lCQURBTkJna3Foa2lHOXcwQkFRc0ZBREFWTVJNd0VRWURWUVFERXdwcmRXSmwKY201bGRHVnpNQjRYRFRJeE1URXlPVEEwTURVME5sb1hEVE14TVRFeU56QTBNRFUwTmxvd0ZURVRNQkVHQTFVRQpBeE1LYTNWaVpYSnVaWFJsY3pDQ0FTSXdEUVlKS29aSWh2Y05BUUVCQlFBRGdnRVBBRENDQVFvQ2dnRUJBSjBuCm94Ulh2Z2VrUjVkb08xRFRWL0VUYXh3MDE4YndmRC9ZS1lnaUM3OEJxZ1diSUpLaHNrd25LSEtEVkxIcGxvRzkKMEVhcXMxS0UxajgwVW1MVE1pV2lPQS84UG5Kd0dyU3plYjZGRzdpT0VxMVdHTVIzV0VuQW9uaDRESXh4Sjg1SwpyamVDVEU2eHFYdGdydHJ6Z2Y3VWx0TEhsZGtZREswVnc3Y08vZmNoRTN6ZzVJNDR1TWxDeGM4QjVZYUkyMURYCmlrZXFtUlJSQUdYbVpmaEwxYUxzaVEveHFETjgyb0J4bHQ0MzFRbEFDQ2VhQWZlNXhBdlNWYjVsWGRGSWxQZisKeTN6aFAvT3N1VTI3UUlENTVRWVllRmhHUmVtTHlNQWxiOTV4SVRkVkNKbVovRHRyNEF3TUR0ekM5VDhvUFRhcwpyS0l2YVBseHhPai9MM0ZmSXBrQ0F3RUFBYU5DTUVBd0RnWURWUjBQQVFIL0JBUURBZ0trTUE4R0ExVWRFd0VCCi93UUZNQU1CQWY4d0hRWURWUjBPQkJZRUZFVUVFamlqUzVHRFQ0Ty9vdlBBdWJLaTdVOElNQTBHQ1NxR1NJYjMKRFFFQkN3VUFBNElCQVFBZnN1Yk55M0ZlbzdlNVVYZzIyK2dxaHJsTDVsM245cXgzNlUxVWd3L3VQbnUweXJySwo3clhTSE9MMWFPejRUM29TWVg1VWttTXN0Q1o0RU8ycmFrUVVSVUl4THBzZ3F1OHlDNHRuRldrN1A0UjBiUlU3CmYvWjlETkU3MDR2Rkl2cHhtVGFUdVhzbDhoaVpBUFRGNER3b25hMURXWkx0Y1QxbHcwTEU4TWdEMVFIZG5iSUoKbGxmRlRwZ1RlMS9uQ3BIYWNOeWU2Wk1zVjBPRUovc1RBRXNkSGRwQXQ0VHJoSHZDM1NBeEFrVFJrUENPSGNwWApKbHh3blNYSGF0OGtlRWlQdHdha2E2RW4wQ1B1VUtKTktHVEdXbEUwVmZ1MWZteW5tTW0xMUt4M1N1NHFIMHJhCkh5citmRUlHSE9ib0FpWEVDSXNNSFprN1VlQlYzd3R3OE9DeAotLS0tLUVORCBDRVJUSUZJQ0FURS0tLS0tCg==
server: https://192.168.59.164:6443
name: cluster1
contexts:
- context:
cluster: cluster1
namespace: default
user: ccx
name: context1
current-context: "context1"
kind: Config
preferences: {}
users:
- name: ccx
user:
client-certificate-data: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSURCakNDQWU2Z0F3SUJBZ0lSQUpudElTVWJSRUplcVhEVjd6MitIUVV3RFFZSktvWklodmNOQVFFTEJRQXcKRlRFVE1CRUdBMVVFQXhNS2EzVmlaWEp1WlhSbGN6QWVGdzB5TVRFeE16QXdNalV3TWpkYUZ3MHlNakV4TXpBdwpNalV3TWpkYU1DQXhFREFPQmdOVkJBb1RCMk5yWVRJd01qRXhEREFLQmdOVkJBTVRBMk5qZURDQ0FTSXdEUVlKCktvWklodmNOQVFFQkJRQURnZ0VQQURDQ0FRb0NnZ0VCQU51QzVmVG1MY3d6b1cwMk95Zll0c1ZZeUtnTDMyaXEKV2ZFYW9UWnlaWUEwdjJDRWsxNlRzdWpqU1Mrc2tIKzR3dGJ6allKSlJSWDd5czVOZXU3VUwvcTEzZmxGeVJjSgpIVDJTcG4xWmNzVEdRMWdWMEJ5S0UwcUdmRXFwdjVKd0QrQWVRNG1HTkJyZFNKNzloTTJTNUIrTlN1WTlnZEtjCmN3QXVSMEswdExHUFJSOVhLNDc0UnZKUC9LcXZ1RTBhYi9qQU1aeXJUVUYxNi9Idnp1Y3lyYjBaOE5PSVdpOUIKTTA0aHZQRC9XOXp5VVRPbHAvREpsSksxUkRidmgwU1RSZHZGSmc0a3o5UThuN1I3MDJMYzJVWTlOdlJQaWtlMgpCSGZqWjBwNXpiYzEwaHhORnRRSnJkeWJXSWNrcTk2RzdoYyttQjhQMzZTdlNLN0luNVlaSTVNQ0F3RUFBYU5HCk1FUXdFd1lEVlIwbEJBd3dDZ1lJS3dZQkJRVUhBd0l3REFZRFZSMFRBUUgvQkFJd0FEQWZCZ05WSFNNRUdEQVcKZ0JSRkJCSTRvMHVSZzArRHY2THp3TG15b3UxUENEQU5CZ2txaGtpRzl3MEJBUXNGQUFPQ0FRRUFKTkUxTnVBcwpvNWNuUlUrNjV5cy8reHhCdDNGZzhETWowSG9rTHVtdkZ0WjdDRGFoWDVmUEh0Nll5bkFmdWxnZGJoYmEwMVVWCmM4NTRzT2ZETzh4WUV1YVJMQ0tuK2EreVlIN1FDT3p0bEpHSGxhS2lZazdKRVJkajB1MTk5Z3BKK0FOTG9TUVAKZkpWd3dmY2xpb0ZmMTZVRFBYTm9jU0pRcmpXaWgzNEhOcnVkQ3kxWGpQT3U3ZXRqVDJJQ1ExTEQwNHc3N0xzLwpzcGVPWUJyT1FGUjMzVXRuMHMveG9HSThFeFB6Q1NwVDFaYy9Kd1JEVW9hRDZMdTgzWGFNRDI0aXArSmoxM1RZCnVJNjIrdTBWZk12cDhlaVM2TWlnd1A3dzd2TUQ2WHF5REYxeVhtc25BRnNwRWh1aUpjRzZmR08yT25rY1JiOFAKUTVjK2lqWTBRRFAzR0E9PQotLS0tLUVORCBDRVJUSUZJQ0FURS0tLS0tCg==
client-key-data: LS0tLS1CRUdJTiBSU0EgUFJJVkFURSBLRVktLS0tLQpNSUlFcEFJQkFBS0NBUUVBMjRMbDlPWXR6RE9oYlRZN0o5aTJ4VmpJcUF2ZmFLcFo4UnFoTm5KbGdEUy9ZSVNUClhwT3k2T05KTDZ5UWY3akMxdk9OZ2tsRkZmdkt6azE2N3RRdityWGQrVVhKRndrZFBaS21mVmx5eE1aRFdCWFEKSElvVFNvWjhTcW0va25BUDRCNURpWVkwR3QxSW52MkV6WkxrSDQxSzVqMkIwcHh6QUM1SFFyUzBzWTlGSDFjcgpqdmhHOGsvOHFxKzRUUnB2K01BeG5LdE5RWFhyOGUvTzV6S3R2Um53MDRoYUwwRXpUaUc4OFA5YjNQSlJNNlduCjhNbVVrclZFTnUrSFJKTkYyOFVtRGlUUDFEeWZ0SHZUWXR6WlJqMDI5RStLUjdZRWQrTm5Tbm5OdHpYU0hFMFcKMUFtdDNKdFloeVNyM29idUZ6NllIdy9mcEs5SXJzaWZsaGtqa3dJREFRQUJBb0lCQVFDaFd2RUtPZ0RFTDllagpYYy9TRkgwVlI1UUg0dUpRSDVpSm9GZU1uRDU4SlVuZit0UVJHMlRSeC9ET09Iem5SYnNESW5pTW9xdEQ0NWhLCldhM1p6T09QMlF2WDVqSlEyb1JCOUlDcGQ0empsQkdBdUZnSUFuNzNzeSs5K2xVMW9XWXFDbFQrekVXVTBjQkcKRG5rR1c4bVFYOTRFcklXM2VRVVh4dXplM2RKTkNYUTdsYmExM0dqRnJkeWNncTd1SktDVUlOZVhrYnVUU09pWgpHeUhwSC9wYm5tdzNxV1lWcXJCMHZydUNVWEV2NkpFUU9wZjVrQmVSRVVGV2owbTB4MFcvUGJOczJUWWRRNDZSCndwVUdzemd6UFE3aTd0R1Q1bVJVeXljem92clRJSzRyZVMyb2VJY0lvdHo3dk5aUFBMMFdDWGVuTlYyUVh0M0YKVEJpK2FJcEJBb0dCQVB6a29VZE1nNXpLUzhEV3VDUUhMaXlCYVpqdnVnR3dvZ2svcFRDc2ZUZlRuYnZISHE3cgpENVF1aXI1Ym92UTRPeDZnUWFpcjJpWURJU0JXQ3grQXFNYnB3cnV2TkpjYWJiWDg3bG0xOThnOUhQU0FBeHZYCldMWmVJRUovZ1lZeXEydGwxU2x3aXlpcUJVKzk1TFdPWkNSMWdQemF4cTY3aHNHZjNCUVh0c0hEQW9HQkFONDEKUjZJOVpzSjcwT2lyNUdORExDMWw2L3lzYk9UMHpkUC9abXA0QzE3ZlZNY1czTDNBOFZ6NkU2bEVDajNJV21NeAovRXdjSWl6Y3RwMmRPdXEvQXQvQkhMK2R3Z2JOaStTT0s5SGNZVWlRczlkZGZmemFLRG9YbU0wZC9Ec0hjemJmCjYvTGRMM2lJaHBmNTBrZU4yOG81RE8vRWVpUGh6OTNyMUtHaElhbnhBb0dCQUtUUitZNzNmanU4Mk54bzFRQ2UKTVdqT08xMXl1RjhMbUwvQVhGQTV5eTZNWEJ3YkxaTkJIaGtzc0Q1YWlQejBmUXQ3MU84eWVlNFdVZ3U2S0d1eQp0YzFXWXhWaG5qdncvSm9FcSsrS0dtREYrODRhZEd3NzBOU2l5aEdzK21UVk4wVzZ5OC9EU3Q4STJJZGRNdWRsClV6MHozQXB3SjllUDNzYnBBazJTR3dsekFvR0FKWkZNY3hsK2JoTVExc1Y0NzI5RDRNa0NoTTJCKzVPSDJQZXQKRVFNS3FSZUk5Yi9md2hVRXVYdHVKNGZoVTJDMEoyRWtEcG9URFJLanJUSVA2L0F2UkRVWjd5dDB5a0dtRFJZYwpRbmJIMjhUYkl1WWpqc1F6V043MGJubExVNCtHNHhnb2cvN3hMNmVsV2J4YTJNNGJBcTF5aW5ibFQzdTE4UWNXCkE5MkNHNEVDZ1lCemVHM2d6akNSWVN3b3ZENXd3WWR6Yi9yRTM5RXpuS2lPVm9xc003MHhWaVhBbjF5NlBGcisKZXBudVA2Mzk1VC8zQzFKUnNJU1cydEdIaFAyN0ZKUHVGKytTZWFheUMvdUZTZEkvWEpLVkx6Y2FrMmoyMHdWOApBTWJVeTNEWlQ2c0JDU2Jhclp3cEM2VUFUUzZHUzQxOWRmWTdJZzdzSHhKM3R2VWNTUUx1N3c9PQotLS0tLUVORCBSU0EgUFJJVkFURSBLRVktLS0tLQo=
[root@master1-163 sefe]#
- 至此呢，kubeconfig文件就准备好了，我们重命名为kc2，并拷贝到客户端主机上吧【之前有个kc1，免得冲突】
[root@master1-163 sefe]# mv kc1 kc2
[root@master1-163 sefe]# scp kc2 192.168.59.156:~
The authenticity of host '192.168.59.156 (192.168.59.156)' can't be established.
ECDSA key fingerprint is SHA256:zRtVBoNePoRXh9aA8eppKwwduS9Rjjr/kT5a7zijzjE.
ECDSA key fingerprint is MD5:b8:53:cc:da:86:2a:97:dc:bd:64:6b:b1:d0:f3:02:ce.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added '192.168.59.156' (ECDSA) to the list of known hosts.
root@192.168.59.156's password:
kc2 100% 5507 4.0MB/s 00:00
[root@master1-163 sefe]#
客户端测试
说明
- 上面我们已经把kc2文件拷贝到当前服务器上了
并且kc2中连接ip是happroxy的ip地址，你确认下是否有误。
所有配置文件内容如下
[root@etcd1 ~]# cat kc2
apiVersion: v1
clusters:
- cluster:
certificate-authority-data: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUM1ekNDQWMrZ0F3SUJBZ0lCQURBTkJna3Foa2lHOXcwQkFRc0ZBREFWTVJNd0VRWURWUVFERXdwcmRXSmwKY201bGRHVnpNQjRYRFRJeE1URXlPVEEwTURVME5sb1hEVE14TVRFeU56QTBNRFUwTmxvd0ZURVRNQkVHQTFVRQpBeE1LYTNWaVpYSnVaWFJsY3pDQ0FTSXdEUVlKS29aSWh2Y05BUUVCQlFBRGdnRVBBRENDQVFvQ2dnRUJBSjBuCm94Ulh2Z2VrUjVkb08xRFRWL0VUYXh3MDE4YndmRC9ZS1lnaUM3OEJxZ1diSUpLaHNrd25LSEtEVkxIcGxvRzkKMEVhcXMxS0UxajgwVW1MVE1pV2lPQS84UG5Kd0dyU3plYjZGRzdpT0VxMVdHTVIzV0VuQW9uaDRESXh4Sjg1SwpyamVDVEU2eHFYdGdydHJ6Z2Y3VWx0TEhsZGtZREswVnc3Y08vZmNoRTN6ZzVJNDR1TWxDeGM4QjVZYUkyMURYCmlrZXFtUlJSQUdYbVpmaEwxYUxzaVEveHFETjgyb0J4bHQ0MzFRbEFDQ2VhQWZlNXhBdlNWYjVsWGRGSWxQZisKeTN6aFAvT3N1VTI3UUlENTVRWVllRmhHUmVtTHlNQWxiOTV4SVRkVkNKbVovRHRyNEF3TUR0ekM5VDhvUFRhcwpyS0l2YVBseHhPai9MM0ZmSXBrQ0F3RUFBYU5DTUVBd0RnWURWUjBQQVFIL0JBUURBZ0trTUE4R0ExVWRFd0VCCi93UUZNQU1CQWY4d0hRWURWUjBPQkJZRUZFVUVFamlqUzVHRFQ0Ty9vdlBBdWJLaTdVOElNQTBHQ1NxR1NJYjMKRFFFQkN3VUFBNElCQVFBZnN1Yk55M0ZlbzdlNVVYZzIyK2dxaHJsTDVsM245cXgzNlUxVWd3L3VQbnUweXJySwo3clhTSE9MMWFPejRUM29TWVg1VWttTXN0Q1o0RU8ycmFrUVVSVUl4THBzZ3F1OHlDNHRuRldrN1A0UjBiUlU3CmYvWjlETkU3MDR2Rkl2cHhtVGFUdVhzbDhoaVpBUFRGNER3b25hMURXWkx0Y1QxbHcwTEU4TWdEMVFIZG5iSUoKbGxmRlRwZ1RlMS9uQ3BIYWNOeWU2Wk1zVjBPRUovc1RBRXNkSGRwQXQ0VHJoSHZDM1NBeEFrVFJrUENPSGNwWApKbHh3blNYSGF0OGtlRWlQdHdha2E2RW4wQ1B1VUtKTktHVEdXbEUwVmZ1MWZteW5tTW0xMUt4M1N1NHFIMHJhCkh5citmRUlHSE9ib0FpWEVDSXNNSFprN1VlQlYzd3R3OE9DeAotLS0tLUVORCBDRVJUSUZJQ0FURS0tLS0tCg==
server: https://192.168.59.164:6443
name: cluster1
contexts:
- context:
cluster: cluster1
namespace: default
user: ccx
name: context1
current-context: "context1"
kind: Config
preferences: {}
users:
- name: ccx
user:
client-certificate-data: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSURCakNDQWU2Z0F3SUJBZ0lSQUpudElTVWJSRUplcVhEVjd6MitIUVV3RFFZSktvWklodmNOQVFFTEJRQXcKRlRFVE1CRUdBMVVFQXhNS2EzVmlaWEp1WlhSbGN6QWVGdzB5TVRFeE16QXdNalV3TWpkYUZ3MHlNakV4TXpBdwpNalV3TWpkYU1DQXhFREFPQmdOVkJBb1RCMk5yWVRJd01qRXhEREFLQmdOVkJBTVRBMk5qZURDQ0FTSXdEUVlKCktvWklodmNOQVFFQkJRQURnZ0VQQURDQ0FRb0NnZ0VCQU51QzVmVG1MY3d6b1cwMk95Zll0c1ZZeUtnTDMyaXEKV2ZFYW9UWnlaWUEwdjJDRWsxNlRzdWpqU1Mrc2tIKzR3dGJ6allKSlJSWDd5czVOZXU3VUwvcTEzZmxGeVJjSgpIVDJTcG4xWmNzVEdRMWdWMEJ5S0UwcUdmRXFwdjVKd0QrQWVRNG1HTkJyZFNKNzloTTJTNUIrTlN1WTlnZEtjCmN3QXVSMEswdExHUFJSOVhLNDc0UnZKUC9LcXZ1RTBhYi9qQU1aeXJUVUYxNi9Idnp1Y3lyYjBaOE5PSVdpOUIKTTA0aHZQRC9XOXp5VVRPbHAvREpsSksxUkRidmgwU1RSZHZGSmc0a3o5UThuN1I3MDJMYzJVWTlOdlJQaWtlMgpCSGZqWjBwNXpiYzEwaHhORnRRSnJkeWJXSWNrcTk2RzdoYyttQjhQMzZTdlNLN0luNVlaSTVNQ0F3RUFBYU5HCk1FUXdFd1lEVlIwbEJBd3dDZ1lJS3dZQkJRVUhBd0l3REFZRFZSMFRBUUgvQkFJd0FEQWZCZ05WSFNNRUdEQVcKZ0JSRkJCSTRvMHVSZzArRHY2THp3TG15b3UxUENEQU5CZ2txaGtpRzl3MEJBUXNGQUFPQ0FRRUFKTkUxTnVBcwpvNWNuUlUrNjV5cy8reHhCdDNGZzhETWowSG9rTHVtdkZ0WjdDRGFoWDVmUEh0Nll5bkFmdWxnZGJoYmEwMVVWCmM4NTRzT2ZETzh4WUV1YVJMQ0tuK2EreVlIN1FDT3p0bEpHSGxhS2lZazdKRVJkajB1MTk5Z3BKK0FOTG9TUVAKZkpWd3dmY2xpb0ZmMTZVRFBYTm9jU0pRcmpXaWgzNEhOcnVkQ3kxWGpQT3U3ZXRqVDJJQ1ExTEQwNHc3N0xzLwpzcGVPWUJyT1FGUjMzVXRuMHMveG9HSThFeFB6Q1NwVDFaYy9Kd1JEVW9hRDZMdTgzWGFNRDI0aXArSmoxM1RZCnVJNjIrdTBWZk12cDhlaVM2TWlnd1A3dzd2TUQ2WHF5REYxeVhtc25BRnNwRWh1aUpjRzZmR08yT25rY1JiOFAKUTVjK2lqWTBRRFAzR0E9PQotLS0tLUVORCBDRVJUSUZJQ0FURS0tLS0tCg==
client-key-data: LS0tLS1CRUdJTiBSU0EgUFJJVkFURSBLRVktLS0tLQpNSUlFcEFJQkFBS0NBUUVBMjRMbDlPWXR6RE9oYlRZN0o5aTJ4VmpJcUF2ZmFLcFo4UnFoTm5KbGdEUy9ZSVNUClhwT3k2T05KTDZ5UWY3akMxdk9OZ2tsRkZmdkt6azE2N3RRdityWGQrVVhKRndrZFBaS21mVmx5eE1aRFdCWFEKSElvVFNvWjhTcW0va25BUDRCNURpWVkwR3QxSW52MkV6WkxrSDQxSzVqMkIwcHh6QUM1SFFyUzBzWTlGSDFjcgpqdmhHOGsvOHFxKzRUUnB2K01BeG5LdE5RWFhyOGUvTzV6S3R2Um53MDRoYUwwRXpUaUc4OFA5YjNQSlJNNlduCjhNbVVrclZFTnUrSFJKTkYyOFVtRGlUUDFEeWZ0SHZUWXR6WlJqMDI5RStLUjdZRWQrTm5Tbm5OdHpYU0hFMFcKMUFtdDNKdFloeVNyM29idUZ6NllIdy9mcEs5SXJzaWZsaGtqa3dJREFRQUJBb0lCQVFDaFd2RUtPZ0RFTDllagpYYy9TRkgwVlI1UUg0dUpRSDVpSm9GZU1uRDU4SlVuZit0UVJHMlRSeC9ET09Iem5SYnNESW5pTW9xdEQ0NWhLCldhM1p6T09QMlF2WDVqSlEyb1JCOUlDcGQ0empsQkdBdUZnSUFuNzNzeSs5K2xVMW9XWXFDbFQrekVXVTBjQkcKRG5rR1c4bVFYOTRFcklXM2VRVVh4dXplM2RKTkNYUTdsYmExM0dqRnJkeWNncTd1SktDVUlOZVhrYnVUU09pWgpHeUhwSC9wYm5tdzNxV1lWcXJCMHZydUNVWEV2NkpFUU9wZjVrQmVSRVVGV2owbTB4MFcvUGJOczJUWWRRNDZSCndwVUdzemd6UFE3aTd0R1Q1bVJVeXljem92clRJSzRyZVMyb2VJY0lvdHo3dk5aUFBMMFdDWGVuTlYyUVh0M0YKVEJpK2FJcEJBb0dCQVB6a29VZE1nNXpLUzhEV3VDUUhMaXlCYVpqdnVnR3dvZ2svcFRDc2ZUZlRuYnZISHE3cgpENVF1aXI1Ym92UTRPeDZnUWFpcjJpWURJU0JXQ3grQXFNYnB3cnV2TkpjYWJiWDg3bG0xOThnOUhQU0FBeHZYCldMWmVJRUovZ1lZeXEydGwxU2x3aXlpcUJVKzk1TFdPWkNSMWdQemF4cTY3aHNHZjNCUVh0c0hEQW9HQkFONDEKUjZJOVpzSjcwT2lyNUdORExDMWw2L3lzYk9UMHpkUC9abXA0QzE3ZlZNY1czTDNBOFZ6NkU2bEVDajNJV21NeAovRXdjSWl6Y3RwMmRPdXEvQXQvQkhMK2R3Z2JOaStTT0s5SGNZVWlRczlkZGZmemFLRG9YbU0wZC9Ec0hjemJmCjYvTGRMM2lJaHBmNTBrZU4yOG81RE8vRWVpUGh6OTNyMUtHaElhbnhBb0dCQUtUUitZNzNmanU4Mk54bzFRQ2UKTVdqT08xMXl1RjhMbUwvQVhGQTV5eTZNWEJ3YkxaTkJIaGtzc0Q1YWlQejBmUXQ3MU84eWVlNFdVZ3U2S0d1eQp0YzFXWXhWaG5qdncvSm9FcSsrS0dtREYrODRhZEd3NzBOU2l5aEdzK21UVk4wVzZ5OC9EU3Q4STJJZGRNdWRsClV6MHozQXB3SjllUDNzYnBBazJTR3dsekFvR0FKWkZNY3hsK2JoTVExc1Y0NzI5RDRNa0NoTTJCKzVPSDJQZXQKRVFNS3FSZUk5Yi9md2hVRXVYdHVKNGZoVTJDMEoyRWtEcG9URFJLanJUSVA2L0F2UkRVWjd5dDB5a0dtRFJZYwpRbmJIMjhUYkl1WWpqc1F6V043MGJubExVNCtHNHhnb2cvN3hMNmVsV2J4YTJNNGJBcTF5aW5ibFQzdTE4UWNXCkE5MkNHNEVDZ1lCemVHM2d6akNSWVN3b3ZENXd3WWR6Yi9yRTM5RXpuS2lPVm9xc003MHhWaVhBbjF5NlBGcisKZXBudVA2Mzk1VC8zQzFKUnNJU1cydEdIaFAyN0ZKUHVGKytTZWFheUMvdUZTZEkvWEpLVkx6Y2FrMmoyMHdWOApBTWJVeTNEWlQ2c0JDU2Jhclp3cEM2VUFUUzZHUzQxOWRmWTdJZzdzSHhKM3R2VWNTUUx1N3c9PQotLS0tLUVORCBSU0EgUFJJVkFURSBLRVktLS0tLQo=
[root@etcd1 ~]#
- 为什么要连接到happroxy，因为我们在happroxy上配置了，只要访问当前ip的6443端口，就会自动转发到master上
我们在happroxy上查看规则
[root@haproxy-164 haproxy]# netstat -ntlp | grep 6443
tcp 0 0 0.0.0.0:6443 0.0.0.0:* LISTEN 2504/haproxy
[root@haproxy-164 haproxy]# tail /etc/haproxy/haproxy.cfg
server app1 127.0.0.1:5001 check
server app2 127.0.0.1:5002 check
server app3 127.0.0.1:5003 check
server app4 127.0.0.1:5004 check
listen k8s-lb *:6443
mode tcp
balance roundrobin
server s1 192.168.59.163:6443 weight 1
server s2 192.168.59.162:6443 weight 1
[root@haproxy-164 haproxy]#
测试
- 如下，没有kubectl命令的话，需要先安装一个命令哈
[root@etcd1 ~]# export KUBECONFIG=kc2
[root@etcd1 ~]#
[root@etcd1 ~]# kubectl get nodes
-bash: kubectl: 未找到命令
[root@etcd1 ~]#
# 我是离线安装的
[root@etcd1 kubelet]# rpm -ivhU * --nodeps --force
警告：3944a45bec4c99d3489993e3642b63972b62ed0a4ccb04cc7655ce0467fddfef-kubectl-1.21.1-0.x86_64.rpm: 头V4 RSA/SHA512 Signature, 密钥 ID 3e1ba8d5: NOKEY
准备中... ################################# [100%]
正在升级/安装...
1:socat-1.7.3.2-2.el7 ################################# [ 9%]
2:libnetfilter_queue-1.0.2-2.el7_2 ################################# [ 18%]
3:libnetfilter_cttimeout-1.0.0-7.el################################# [ 27%]
4:libnetfilter_cthelper-1.0.0-11.el################################# [ 36%]
5:libnetfilter_conntrack-1.0.6-1.el################################# [ 45%]
6:conntrack-tools-1.4.4-7.el7 ################################# [ 55%]
7:kubernetes-cni-0.8.7-0 ################################# [ 64%]
8:kubelet-1.21.1-0 ################################# [ 73%]
9:cri-tools-1.19.0-0 ################################# [ 82%]
10:kubectl-1.21.1-0 ################################# [ 91%]
11:kubeadm-1.21.1-0 ################################# [100%]
[root@etcd1 kubelet]# systemctl enable kubelet.service --now
Created symlink from /etc/systemd/system/multi-user.target.wants/kubelet.service to /usr/lib/systemd/system/kubelet.service.
[root@etcd1 kubelet]#
- 再来，测试
没问题，完美啊，卧槽~ 一次性成功了 这次没有处理报错 好感动啊
[root@etcd1 kubelet]# cd
[root@etcd1 ~]# export KUBECONFIG=kc2
[root@etcd1 ~]# kubectl get nodes
NAME STATUS ROLES AGE VERSION
master1-163 Ready control-plane,master 23h v1.21.1
master2-162 Ready control-plane,master 22h v1.21.1
worker-165 Ready <none> 18h v1.21.1
[root@etcd1 ~]#
[root@etcd1 ~]# kubectl get ns
NAME STATUS AGE
default Active 23h
kube-node-lease Active 23h
kube-public Active 23h
kube-system Active 23h
ns1 Active 100m
ns2 Active 99m
[root@etcd1 ~]#
[root@etcd1 ~]# kubectl create ns ns3
namespace/ns3 created
[root@etcd1 ~]#
[root@etcd1 ~]# kubectl get ns
NAME STATUS AGE
default Active 23h
kube-node-lease Active 23h
kube-public Active 23h
kube-system Active 23h
ns1 Active 100m
ns2 Active 99m
ns3 Active 6s
[root@etcd1 ~]#
更好的部署master高可用的方式
-
这是国内的一个方式，叫做
sealos收费的，但是一年就几十块钱，很便宜，而且一件部署，炒鸡方便啊~ 上面手动部署的太麻烦了，如果不想折腾的，那么sealos绝对是你的首选哦 -
官网已经做了很详细的使用流程了，而且很简单
我是干技术的，所以呢我还是喜欢使用上面自己动手的配置方法，我不会使用这个，所以我这就不多累赘了，感兴趣的小伙伴去官网注册试试把 ~
标签：haproxy,master1,流程,192.168,kubeconfig,sefe,kube,root,163 来源： https://blog.csdn.net/cuichongxin/article/details/121626097
本站声明： 1. iCode9 技术分享网（下文简称本站）提供的所有内容，仅供技术学习、探讨和分享； 2. 关于本站的所有留言、评论、转载及引用，纯属内容发起人的个人观点，与本站观点和立场无关； 3. 关于本站的所有言论和文字，纯属内容发起人的个人观点，与本站观点和立场无关； 4. 本站文章均是网友提供，不完全保证技术分享内容的完整性、准确性、时效性、风险性和版权归属；如您发现该文章侵犯了您的权益，可联系我们第一时间进行删除； 5. 本站为非盈利性的个人网站，所有内容不会用来进行牟利，也不会利用任何形式的广告来间接获益，纯粹是为了广大技术爱好者提供技术内容和技术思想的分享性交流网站。