4-6 k8s构建zookeeper集群
·
前言
ZooKeeper是一个分布式的,开放源码的分布式应用程序协调服务,是Google的Chubby一个开源的实现,是Hadoop和Hbase的重要组件。它是一个为分布式应用提供一致性服务的软件,提供的功能包括:配置维护、域名服务、分布式同步、组服务等。
下面我们通过k8s构建zookeeper镜像,上传私有仓库,创建PV/PVC,最后yaml部署3个节点的zookeeper集群。
集群需要奇数位,本文创建了3个pv,3个pvc,3个service,3个pod。
zookeeper镜像
准备文件,包括安装包,启动脚本,配置,秘钥等:
ls zookeeper/
# 目录,包括查看zk状态脚本
bin
# 目录,包括zoo配置和属性
conf
# 构建文件
Dockerfile
# 启动脚本,动态生成zoo id
entrypoint.sh
# 秘钥文件
KEYS
# 镜像源
repositories
# zk安装包
zookeeper-3.4.14.tar.gz
# 安装包验证
zookeeper-3.4.14.tar.gz.asc
构建Dockerfile镜像内容:
vim Dockerfile
# 基于官网精简版java8
FROM elevy/slim_java:8
ENV ZK_VERSION 3.4.14
ADD repositories /etc/apk/repositories
# 拷贝zookeeper程序
COPY zookeeper-3.4.14.tar.gz /tmp/zk.tgz
COPY zookeeper-3.4.14.tar.gz.asc /tmp/zk.tgz.asc
COPY KEYS /tmp/KEYS
RUN apk add --no-cache --virtual .build-deps \
...
...
# 复制配置和启动文件,一定要加上可执行权限。
COPY conf /zookeeper/conf/
COPY bin/zkReady.sh /zookeeper/bin/
RUN chmod +x /zookeeper/bin/zkReady.sh
COPY entrypoint.sh /
RUN chmod +x /entrypoint.sh
# 环境变量
ENV PATH=/zookeeper/bin:${PATH} \
ZOO_LOG_DIR=/zookeeper/log \
ZOO_LOG4J_PROP="INFO, CONSOLE, ROLLINGFILE" \
JMXPORT=9010
# 启动脚本ENTRYPOINT 参数为CMD
ENTRYPOINT [ "/entrypoint.sh" ]
CMD [ "zkServer.sh", "start-foreground" ]
# 暴露服务端口
EXPOSE 2181 2888 3888 9010
构建镜像,上传镜像:
nerdctl build -t easzlab.io.local:5000/myhub/zookeeper:v1 .
nerdctl push --insecure-registry easzlab.io.local:5000/myhub/zookeeper:v1
创建NFS共享目录
在NFS服务器创建3个共享目录:
# 创建3个共享目录,属主属组选nfsnobody。
mkdir -p /data/k8sdata/myserver/zookeeper-datadir-1
mkdir -p /data/k8sdata/myserver/zookeeper-datadir-2
mkdir -p /data/k8sdata/myserver/zookeeper-datadir-3
chown nfsnobody.nfsnobody /data/k8sdata/myserver/zookeeper-datadir-1
chown nfsnobody.nfsnobody /data/k8sdata/myserver/zookeeper-datadir-2
chown nfsnobody.nfsnobody /data/k8sdata/myserver/zookeeper-datadir-3
# 提供共享服务,可共享的IP段为物理机IP,rw可读写,all_squash远程用户连到nfs服务器都压榨成nfsnobody用户。
cat << EOF >> /etc/exports
/data/k8sdata/myserver/zookeeper-datadir-1 192.168.100.0/24(rw,all_squash)
/data/k8sdata/myserver/zookeeper-datadir-2 192.168.100.0/24(rw,all_squash)
/data/k8sdata/myserver/zookeeper-datadir-3 192.168.100.0/24(rw,all_squash)
EOF
# 重新加载,查看挂载情况
exportfs -r
exportfs -v
# nfs涉及端口较多,测试环境可关闭防火墙
systemctl disable --now firewalld
创建PV
创建3个pv,分配的3个pv总和可以比NFS硬盘空间大,但不建议。
---
apiVersion: v1
kind: PersistentVolume
metadata:
name: zookeeper-datadir-pv-1
spec:
capacity:
storage: 20Gi
accessModes:
- ReadWriteOnce
nfs:
server: 192.168.100.155
path: /data/k8sdata/myserver/zookeeper-datadir-1
---
apiVersion: v1
kind: PersistentVolume
metadata:
name: zookeeper-datadir-pv-2
spec:
capacity:
storage: 20Gi
accessModes:
- ReadWriteOnce
nfs:
server: 192.168.100.155
path: /data/k8sdata/myserver/zookeeper-datadir-2
---
apiVersion: v1
kind: PersistentVolume
metadata:
name: zookeeper-datadir-pv-3
spec:
capacity:
storage: 20Gi
accessModes:
- ReadWriteOnce
nfs:
server: 192.168.100.155
path: /data/k8sdata/myserver/zookeeper-datadir-3
查看创建的3个pv:
sudo kubectl get pv
NAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS CLAIM STORAGECLASS REASON AGE
zookeeper-datadir-pv-1 20Gi RWO Retain Available 7s
zookeeper-datadir-pv-2 20Gi RWO Retain Available 7s
zookeeper-datadir-pv-3 20Gi RWO Retain Available 7s
创建PVC
创建3个pvc,pvc容量不能大于对应的pv容量:
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: zookeeper-datadir-pvc-1
namespace: myserver
spec:
accessModes:
- ReadWriteOnce
volumeName: zookeeper-datadir-pv-1
resources:
requests:
storage: 10Gi
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: zookeeper-datadir-pvc-2
namespace: myserver
spec:
accessModes:
- ReadWriteOnce
volumeName: zookeeper-datadir-pv-2
resources:
requests:
storage: 10Gi
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: zookeeper-datadir-pvc-3
namespace: myserver
spec:
accessModes:
- ReadWriteOnce
volumeName: zookeeper-datadir-pv-3
resources:
requests:
storage: 10Gi
查看3个pvc:
sudo kubectl get pvc -n myserver
NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE
zookeeper-datadir-pvc-1 Bound zookeeper-datadir-pv-1 20Gi RWO 47s
zookeeper-datadir-pvc-2 Bound zookeeper-datadir-pv-2 20Gi RWO 47s
zookeeper-datadir-pvc-3 Bound zookeeper-datadir-pv-3 20Gi RWO 46s
部署zookeeper集群
创建服务:zookeeper,zookeeper1,zookeeper2,zookeeper3。
创建Pod:zookeeper1,zookeeper2,zookeeper3。
服务端口:2181,2888,3888。
apiVersion: v1
kind: Service
metadata:
name: zookeeper
namespace: myserver
spec:
ports:
- name: client
port: 2181
selector:
app: zookeeper
---
apiVersion: v1
kind: Service
metadata:
name: zookeeper1
namespace: myserver
spec:
type: NodePort
ports:
- name: client
port: 2181
nodePort: 32181
- name: followers
port: 2888
- name: election
port: 3888
selector:
app: zookeeper
server-id: "1"
---
apiVersion: v1
kind: Service
metadata:
name: zookeeper2
namespace: myserver
spec:
type: NodePort
ports:
- name: client
port: 2181
nodePort: 32182
- name: followers
port: 2888
- name: election
port: 3888
selector:
app: zookeeper
server-id: "2"
---
apiVersion: v1
kind: Service
metadata:
name: zookeeper3
namespace: myserver
spec:
type: NodePort
ports:
- name: client
port: 2181
nodePort: 32183
- name: followers
port: 2888
- name: election
port: 3888
selector:
app: zookeeper
server-id: "3"
---
kind: Deployment
apiVersion: apps/v1
metadata:
name: zookeeper1
namespace: myserver
spec:
replicas: 1
selector:
matchLabels:
app: zookeeper
template:
metadata:
labels:
app: zookeeper
server-id: "1"
spec:
volumes:
- name: data
emptyDir: {}
- name: wal
emptyDir:
medium: Memory
containers:
- name: server
image: easzlab.io.local:5000/myhub/zookeeper:v1
imagePullPolicy: Always
env:
- name: MYID
value: "1"
- name: SERVERS
value: "zookeeper1,zookeeper2,zookeeper3"
- name: JVMFLAGS
value: "-Xmx2G"
ports:
- containerPort: 2181
- containerPort: 2888
- containerPort: 3888
volumeMounts:
- mountPath: "/zookeeper/data"
name: zookeeper-datadir-pvc-1
volumes:
- name: zookeeper-datadir-pvc-1
persistentVolumeClaim:
claimName: zookeeper-datadir-pvc-1
---
kind: Deployment
apiVersion: apps/v1
metadata:
name: zookeeper2
namespace: myserver
spec:
replicas: 1
selector:
matchLabels:
app: zookeeper
template:
metadata:
labels:
app: zookeeper
server-id: "2"
spec:
volumes:
- name: data
emptyDir: {}
- name: wal
emptyDir:
medium: Memory
containers:
- name: server
image: easzlab.io.local:5000/myhub/zookeeper:v1
imagePullPolicy: Always
env:
- name: MYID
value: "2"
- name: SERVERS
value: "zookeeper1,zookeeper2,zookeeper3"
- name: JVMFLAGS
value: "-Xmx2G"
ports:
- containerPort: 2181
- containerPort: 2888
- containerPort: 3888
volumeMounts:
- mountPath: "/zookeeper/data"
name: zookeeper-datadir-pvc-2
volumes:
- name: zookeeper-datadir-pvc-2
persistentVolumeClaim:
claimName: zookeeper-datadir-pvc-2
---
kind: Deployment
apiVersion: apps/v1
metadata:
name: zookeeper3
namespace: myserver
spec:
replicas: 1
selector:
matchLabels:
app: zookeeper
template:
metadata:
labels:
app: zookeeper
server-id: "3"
spec:
volumes:
- name: data
emptyDir: {}
- name: wal
emptyDir:
medium: Memory
containers:
- name: server
image: easzlab.io.local:5000/myhub/zookeeper:v1
imagePullPolicy: Always
env:
- name: MYID
value: "3"
- name: SERVERS
value: "zookeeper1,zookeeper2,zookeeper3"
- name: JVMFLAGS
value: "-Xmx2G"
ports:
- containerPort: 2181
- containerPort: 2888
- containerPort: 3888
volumeMounts:
- mountPath: "/zookeeper/data"
name: zookeeper-datadir-pvc-3
volumes:
- name: zookeeper-datadir-pvc-3
persistentVolumeClaim:
claimName: zookeeper-datadir-pvc-3
检查集群
容器启动后检查状态,3个节点一主两从为正常状态:
sudo kubectl get pod -n myserver
zookeeper1-68ff8f4f7d-kc86m 1/1 Running 0 17m
zookeeper2-5bcfcd94df-8qjmw 1/1 Running 0 17m
zookeeper3-7d66b87c88-v6x2j 1/1 Running 0 17m
sudo kubectl exec zookeeper1-68ff8f4f7d-kc86m -n myserver -- /zookeeper/bin/zkServer.sh status | grep Mode
Mode: follower
sudo kubectl exec zookeeper2-5bcfcd94df-8qjmw -n myserver -- /zookeeper/bin/zkServer.sh status | grep Mode
Mode: follower
sudo kubectl exec zookeeper3-7d66b87c88-v6x2j -n myserver -- /zookeeper/bin/zkServer.sh status | grep Mode
Mode: leader
更多推荐


所有评论(0)