NFS-storageClass设置

韩华盛

432人浏览 · 2025-03-19 13:46:28

韩华盛 · 2025-03-19 13:46:28 发布

存储类简介

在k8s中，存储类可以实现存储资源的动态制备，在使用pvc时，不需要事先准备好pv，存储类会根据pvc的信息，动态创建pv。

创建nfs存储类

前提条件

1、准备一个k8s集群，本例集群版本为1.23.6
2、在其中一个node节点创建nfs服务，以下是目录权限和nfs配置详情，假如在创建pvc过程中遇到权限不足报错，可以把“/nfs”权限调成777.

[root@k8s-node2 nfs]# ll -d /nfs
drwxr-xr-x 4 root root 28 Mar 19 09:35 /nfs
[root@k8s-node2 nfs]# cat /etc/exports
/nfs  *(rw,sync,no_all_squash,no_root_squash)
[root@k8s-node2 nfs]#

RBAC配置

apiVersion: v1
kind: ServiceAccount
metadata:
  name: nfs-client-provisioner
  namespace: default
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: nfs-client-provisioner-runner
#定义nfs-client所需要的集群级别权限列表
rules:
- apiGroups: [""]
  resources: ["nodes"]
  verbs: ["get", "list", "watch"]
- apiGroups: [""]
  resources: ["persistentvolumes"]
  verbs: ["get", "list", "watch", "create", "delete"]
- apiGroups: [""]
  resources: ["persistentvolumeclaims"]
  verbs: ["get", "list", "watch", "update",]
- apiGroups: ["storage.k8s.io"]
  resources: ["storageclasses"]
  verbs: ["get", "list", "watch"]
- apiGroups: [""]
  resources: ["events"]
  verbs: ["create", "update", "patch"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: run-nfs-client-provisioner
#将nfs-client-provisioner账号与clusterrole绑定
subjects:
- kind: ServiceAccount
  name: nfs-client-provisioner
  namespace: default
roleRef:
  kind: ClusterRole
  name: nfs-client-provisioner-runner
  apiGroup: rbac.authorization.k8s.io
---
kind: Role
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: leader-locking-nfs-client-provisioner
  namespace: default
rules:
- apiGroups: [""]
  resources: ["endpoints"]
  verbs: ["get", "list", "watch", "create", "update", "patch"]
---
kind: RoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: leader-locking-nfs-client-provisioner
  namespace: default
subjects:
- kind: ServiceAccount
  name: nfs-client-provisioner
  namespace: default
roleRef:
  kind: Role
  name: leader-locking-nfs-client-provisioner
  apiGroup: rbac.authorization.k8s.io

deploy配置

apiVersion: apps/v1
kind: Deployment
metadata:
  name: nfs-client-provisioner
  namespace: default
  labels:
    app: nfs-client-provisioner
spec:
  replicas: 1
  selector:
    matchLabels:
      app: nfs-client-provisioner
  strategy:
    type: Recreate
  template:
    metadata:
      labels:
        app: nfs-client-provisioner
    spec:
      serviceAccountName: nfs-client-provisioner
      containers:
      - name: nfs-client-provisioner
        # 镜像为阿里镜像源
        image: registry.cn-hangzhou.aliyuncs.com/smallsoup/nfs-subdir-external-provisioner:v4.0.2
        volumeMounts:
        - name: nfs-client-root
          mountPath: /persistentvolumes
        env:
        - name: PROVISIONER_NAME
          value: k8s-sigs.io/nfs-subdir-external-provisioner
        - name: NFS_SERVER
          # 该地址为nfs服务器的地址
          value: 192.168.1.43
        - name: NFS_PATH
          # 该路径为nfs服务器提供的路径
          value: /nfs
      volumes:
      - name: nfs-client-root
        nfs:
          # 该地址为nfs服务器的地址
          server: 192.168.1.43
          # 该路径为nfs服务器提供的路径
          path: /nfs
      securityContext:
        # 配置容器访问nfs为root用户权限，不需要此权限可以注释掉
        runAsUser: 0
        runAsGroup: 0

storageClass配置

# nfs-storageclass.yaml
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: nfs-client
# 该名字必须要deployment  env中， PROVISIONER_NAME  值相匹配
provisioner: k8s-sigs.io/nfs-subdir-external-provisioner
parameters:
  archiveOnDelete: "false"
reclaimPolicy: Retain
allowVolumeExpansion: true

通过创建pvc验证storageClass创建成功

kind: PersistentVolumeClaim
apiVersion: v1
metadata:
  name: test-claim
spec:
  # 指定storageclass
  storageClassName: nfs-client 
  accessModes:
  - ReadWriteMany
  resources:
    requests:
      storage: 1Mi

#  使用storage创建pvc成功
[root@k8s-master storageclass]# kubectl get pvc
NAME         STATUS   VOLUME                                     CAPACITY   ACCESS MODES   STORAGECLASS   AGE
test-claim   Bound    pvc-e287becf-99fe-4211-a2e4-eb05d36903c6   1Mi        RWX            nfs-client     3h36m
[root@k8s-master storageclass]# kubectl get pv
NAME                                       CAPACITY   ACCESS MODES   RECLAIM POLICY   STATUS   CLAIM                STORAGECLASS   REASON   AGE
pvc-e287becf-99fe-4211-a2e4-eb05d36903c6   1Mi        RWX            Retain           Bound    default/test-claim   nfs-client              3h36m
[root@k8s-master storageclass]#

排查方法

通过nfs client pod 日志查看错误原因

[root@k8s-master storageclass]# kubectl get pods 
NAME                                      READY   STATUS    RESTARTS       AGE
nfs-client-provisioner-69b65f988f-s6kgj   1/1     Running   1 (2m3s ago)   16h   #nfs client pod
[root@k8s-master storageclass]# kubectl logs pod/nfs-client-provisioner-69b65f988f-s6kgj
I0319 02:01:39.158374       1 leaderelection.go:242] attempting to acquire leader lease  default/k8s-sigs.io-nfs-subdir-external-provisioner...
I0319 02:02:04.897435       1 leaderelection.go:252] successfully acquired lease default/k8s-sigs.io-nfs-subdir-external-provisioner
I0319 02:02:04.897743       1 controller.go:820] Starting provisioner controller k8s-sigs.io/nfs-subdir-external-provisioner_nfs-client-provisioner-69b65f988f-s6kgj_1ecb26ae-10d2-4719-b6fa-db6025fe6e10!
I0319 02:02:04.899589       1 event.go:278] Event(v1.ObjectReference{Kind:"Endpoints", Namespace:"default", Name:"k8s-sigs.io-nfs-subdir-external-provisioner", UID:"77b4dd66-75bb-498f-9b00-260c323a56a3", APIVersion:"v1", ResourceVersion:"4385361", FieldPath:""}): type: 'Normal' reason: 'LeaderElection' nfs-client-provisioner-69b65f988f-s6kgj_1ecb26ae-10d2-4719-b6fa-db6025fe6e10 became leader
I0319 02:02:05.198037       1 controller.go:869] Started provisioner controller k8s-sigs.io/nfs-subdir-external-provisioner_nfs-client-provisioner-69b65f988f-s6kgj_1ecb26ae-10d2-4719-b6fa-db6025fe6e10!
I0319 02:04:09.560285       1 controller.go:1317] provision "default/test-claim" class "nfs-client": started
I0319 02:04:10.096006       1 event.go:278] Event(v1.ObjectReference{Kind:"PersistentVolumeClaim", Namespace:"default", Name:"test-claim", UID:"e287becf-99fe-4211-a2e4-eb05d36903c6", APIVersion:"v1", ResourceVersion:"4385530", FieldPath:""}): type: 'Normal' reason: 'Provisioning' External provisioner is provisioning volume for claim "default/test-claim"
I0319 02:04:10.098276       1 controller.go:1420] provision "default/test-claim" class "nfs-client": volume "pvc-e287becf-99fe-4211-a2e4-eb05d36903c6" provisioned
I0319 02:04:10.098306       1 controller.go:1437] provision "default/test-claim" class "nfs-client": succeeded
I0319 02:04:10.098315       1 volume_store.go:212] Trying to save persistentvolume "pvc-e287becf-99fe-4211-a2e4-eb05d36903c6"
I0319 02:04:13.618716       1 volume_store.go:219] persistentvolume "pvc-e287becf-99fe-4211-a2e4-eb05d36903c6" saved
I0319 02:04:13.619244       1 event.go:278] Event(v1.ObjectReference{Kind:"PersistentVolumeClaim", Namespace:"default", Name:"test-claim", UID:"e287becf-99fe-4211-a2e4-eb05d36903c6", APIVersion:"v1", ResourceVersion:"4385530", FieldPath:""}): type: 'Normal' reason: 'ProvisioningSucceeded' Successfully provisioned volume pvc-e287becf-99fe-4211-a2e4-eb05d36903c6
[root@k8s-master storageclass]#

查看pvc状态
通过查看pvc event信息查看报错原因

[root@k8s-master storageclass]# kubectl get pvc
NAME         STATUS   VOLUME                                     CAPACITY   ACCESS MODES   STORAGECLASS   AGE
test-claim   Bound    pvc-e287becf-99fe-4211-a2e4-eb05d36903c6   1Mi        RWX            nfs-client     8m52s
[root@k8s-master storageclass]# kubectl describe pvc/test-claim
Name:          test-claim
Namespace:     default
StorageClass:  nfs-client
Status:        Bound
Volume:        pvc-e287becf-99fe-4211-a2e4-eb05d36903c6
Labels:        <none>
Annotations:   pv.kubernetes.io/bind-completed: yes
               pv.kubernetes.io/bound-by-controller: yes
               volume.beta.kubernetes.io/storage-provisioner: k8s-sigs.io/nfs-subdir-external-provisioner
               volume.kubernetes.io/storage-provisioner: k8s-sigs.io/nfs-subdir-external-provisioner
Finalizers:    [kubernetes.io/pvc-protection]
Capacity:      1Mi
Access Modes:  RWX
VolumeMode:    Filesystem
Used By:       <none>
Events:
  Type    Reason                 Age                    From                                                                                                                      Message
  ----    ------                 ----                   ----                                                                                                                      -------
  Normal  ExternalProvisioning   7m30s (x2 over 7m30s)  persistentvolume-controller                                                                                               waiting for a volume to be created, either by external provisioner "k8s-sigs.io/nfs-subdir-external-provisioner" or manually created by system administrator
  Normal  Provisioning           7m29s                  k8s-sigs.io/nfs-subdir-external-provisioner_nfs-client-provisioner-69b65f988f-s6kgj_1ecb26ae-10d2-4719-b6fa-db6025fe6e10  External provisioner is provisioning volume for claim "default/test-claim"
  Normal  ProvisioningSucceeded  7m26s                  k8s-sigs.io/nfs-subdir-external-provisioner_nfs-client-provisioner-69b65f988f-s6kgj_1ecb26ae-10d2-4719-b6fa-db6025fe6e10  Successfully provisioned volume pvc-e287becf-99fe-4211-a2e4-eb05d36903c6
[root@k8s-master storageclass]#

设置和取消默认storageclass

[root@k8s-master storageclass]# kubectl get storageclass
NAME         PROVISIONER                                   RECLAIMPOLICY   VOLUMEBINDINGMODE   ALLOWVOLUMEEXPANSION   AGE
nfs-client   k8s-sigs.io/nfs-subdir-external-provisioner   Retain          Immediate           true                   19h
#  设置nfs-client存储类为默认存储类，nfs-client替换为实际存储类名称
[root@k8s-master storageclass]# kubectl patch storageclass nfs-client -p '{"metadata": {"annotations":{"storageclass.kubernetes.io/is-default-class":"true"}}}'
storageclass.storage.k8s.io/nfs-client patched
# 设置默认存储类成功，存储类名后面有（default）表示设置成功
[root@k8s-master storageclass]# kubectl get storageclass
NAME                   PROVISIONER                                   RECLAIMPOLICY   VOLUMEBINDINGMODE   ALLOWVOLUMEEXPANSION   AGE
nfs-client (default)   k8s-sigs.io/nfs-subdir-external-provisioner   Retain          Immediate           true                   20h
# 取消默认存储类
[root@k8s-master storageclass]# kubectl patch storageclass nfs-client -p '{"metadata": {"annotations":{"storageclass.kubernetes.io/is-default-class":"false"}}}'
storageclass.storage.k8s.io/nfs-client patched
[root@k8s-master storageclass]# kubectl get storageclass
NAME         PROVISIONER                                   RECLAIMPOLICY   VOLUMEBINDINGMODE   ALLOWVOLUMEEXPANSION   AGE
nfs-client   k8s-sigs.io/nfs-subdir-external-provisioner   Retain          Immediate           true                   20h
[root@k8s-master storageclass]#

乐奇 Rokid 开放社区

更多推荐

TestStar | 接入 AIUI 测试 30 天后，别拿别人的数算自己的账

乐奇 Rokid 开放社区

Meta 发布个人 AI 智能体 Muse：能替你办事，但安全这关还没过

北京时间 9 月 9 日，Meta 正式发布旗下首款个人 AI 智能体（内部代号 Hatch）。据路透社、IT之家等报道，它不再只是一个"陪聊"的对话助手，而是能自主代表用户完成等真实任务。初期仅在美国上线，可通过专门的 Muse 应用或 WhatsApp 使用，Meta 称"很快"还会把它接入智能眼镜产品线。