一、ETCD是什么?

etcd 是一个分布式可靠的键值存储系统,用于分布式系统中最关键的数据,主要特点包括:

  • 简单:具有明确的、面向用户的 API(gRPC)

  • 安全:自动 TLS 支持,并可选的客户端证书认证

  • 快速:经过基准测试,支持每秒 10,000 次写入

  • 可靠:使用 Raft 算法进行适当分布

    etcd 使用 Go 语言编写,并利用 Raft 共识算法来管理高度可用的复制日志。

​ 许多公司在生产环境中使用 etcd,开发团队在关键部署场景中支持它,etcd 经常与 Kubernetes、locksmith、vulcand、Doorman 等应用程序配合使用。通过严格的健壮性测试,进一步确保了系统的可靠性。

二、ETCD的集群环境如何部署?

1、主机资源准备

主机ipetcd服务名CPU内存
192.168.188.101s14核4G
192.168.188.102s24核4G
192.168.188.103s34核4G

设置免密登录:

在三台机器都执行执行:

ssh-keygen -t rsa -b 4096
ssh-copy-id root@192.168.188.101
ssh-copy-id root@192.168.188.102
ssh-copy-id root@192.168.188.102

2、借助play.etcd.io官网来进行etcd集群部署

地址:http://play.etcd.io/install#top

3、生成自签名的证书

a)、在页面按自己主机信息输出相关的信息

打开的页面如下:
在这里插入图片描述

修改ip地址、端口信息以及etcd的数据目录信息。

在这里插入图片描述

b)、安装cfssl工具(三台主机上都要操作)

cfssl的安装脚本如下:

#!/bin/bash
rm -f /tmp/cfssl* && rm -rf /tmp/certs && mkdir -p /tmp/certs

curl -L https://pkg.cfssl.org/R1.2/cfssl_linux-amd64 -o /tmp/cfssl
chmod +x /tmp/cfssl
sudo mv /tmp/cfssl /usr/local/bin/cfssl

curl -L https://pkg.cfssl.org/R1.2/cfssljson_linux-amd64 -o /tmp/cfssljson
chmod +x /tmp/cfssljson
sudo mv /tmp/cfssljson /usr/local/bin/cfssljson

/usr/local/bin/cfssl version
/usr/local/bin/cfssljson -h

mkdir -p /data/etcd/certs
c)、生成自签名的CA根证书(三台主机都要进行操作)

在这里插入图片描述

生成根证书的脚本(etcd-root-ca_gencert.sh)如下:

#!/bin/bash
mkdir -p /data/etcd/certs

cat > /data/etcd/certs/etcd-root-ca-csr.json <<EOF
{
  "key": {
    "algo": "rsa",
    "size": 2048
  },
  "names": [
    {
      "O": "etcd",
      "OU": "etcd Security",
      "L": "shengzhen",
      "ST": "guangdong",
      "C": "CN"
    }
  ],
  "CN": "etcd-root-ca"
}
EOF
cfssl gencert --initca=true /data/etcd/certs/etcd-root-ca-csr.json | cfssljson --bare /data/etcd/certs/etcd-root-ca

# verify
openssl x509 -in /data/etcd/certs/etcd-root-ca.pem -text -noout


# cert-generation configuration
cat > /data/etcd/certs/etcd-gencert.json <<EOF
{
  "signing": {
    "default": {
        "usages": [
          "signing",
          "key encipherment",
          "server auth",
          "client auth"
        ],
        "expiry": "87600h"
    }
  }
}
EOF

执行操作:

[root@localhost ~]# vi etcd-root-ca_gencert.sh
[root@localhost ~]# chmod +x etcd-root-ca_gencert.sh
[root@localhost ~]# ./etcd-root-ca_gencert.sh

将根证书信息同步到102、103机器上

scp  /data/etcd/certs/* root@192.168.188.102:/data/etcd/certs/
scp  /data/etcd/certs/* root@192.168.188.103:/data/etcd/certs/
d)、生成101机器上的私有证书

private_key_install.sh

#!/bin/bash
mkdir -p /data/etcd/certs

cat > /data/etcd/certs/s1-ca-csr.json <<EOF
{
  "key": {
    "algo": "rsa",
    "size": 2048
  },
  "names": [
    {
      "O": "etcd",
      "OU": "etcd Security",
      "L": "shengzhen",
      "ST": "guangdong",
      "C": "CN"
    }
  ],
  "CN": "s1",
  "hosts": [
    "127.0.0.1",
    "localhost",
    "192.168.188.101"
  ]
}
EOF
cfssl gencert \
  --ca /data/etcd/certs/etcd-root-ca.pem \
  --ca-key /data/etcd/certs/etcd-root-ca-key.pem \
  --config /data/etcd/certs/etcd-gencert.json \
  /data/etcd/certs/s1-ca-csr.json | cfssljson --bare /data/etcd/certs/s1

# verify
openssl x509 -in /data/etcd/certs/s1.pem -text -noout
e)、生成102机器上的私有证书
#!/bin/bash
mkdir -p /data/etcd/certs

cat > /data/etcd/certs/s2-ca-csr.json <<EOF
{
  "key": {
    "algo": "rsa",
    "size": 2048
  },
  "names": [
    {
      "O": "etcd",
      "OU": "etcd Security",
      "L": "shengzhen",
      "ST": "guangdong",
      "C": "CN"
    }
  ],
  "CN": "s2",
  "hosts": [
    "127.0.0.1",
    "localhost",
    "192.168.188.102"
  ]
}
EOF
cfssl gencert \
  --ca /data/etcd/certs/etcd-root-ca.pem \
  --ca-key /data/etcd/certs/etcd-root-ca-key.pem \
  --config /data/etcd/certs/etcd-gencert.json \
  /data/etcd/certs/s2-ca-csr.json | cfssljson --bare /data/etcd/certs/s2

# verify
openssl x509 -in /data/etcd/certs/s2.pem -text -noout
f)、生成103机器上的私有证书
#!/bin/bash
mkdir -p /data/etcd/certs

cat > /data/etcd/certs/s3-ca-csr.json <<EOF
{
  "key": {
    "algo": "rsa",
    "size": 2048
  },
  "names": [
    {
      "O": "etcd",
      "OU": "etcd Security",
      "L": "shengzhen",
      "ST": "guangdong",
      "C": "CN"
    }
  ],
  "CN": "s3",
  "hosts": [
    "127.0.0.1",
    "localhost",
    "192.168.188.103"
  ]
}
EOF
cfssl gencert \
  --ca /data/etcd/certs/etcd-root-ca.pem \
  --ca-key /data/etcd/certs/etcd-root-ca-key.pem \
  --config /data/etcd/certs/etcd-gencert.json \
  /data/etcd/certs/s3-ca-csr.json | cfssljson --bare /data/etcd/certs/s3

# verify
openssl x509 -in /data/etcd/certs/s3.pem -text -noout

4、部署ETCD

0)、下载etcd(三台服务上都要操作)

etcd_down.sh

#!/bin/bash
ETCD_VER=v3.3.8

# choose either URL
GOOGLE_URL=https://storage.googleapis.com/etcd
GITHUB_URL=https://github.com/coreos/etcd/releases/download
DOWNLOAD_URL=${GOOGLE_URL}

rm -f /tmp/etcd-${ETCD_VER}-linux-amd64.tar.gz
rm -rf /data/etcd-install && mkdir -p /data/etcd-install
rm -rf /tmp/test-etcd  && mkdir -p /tmp/test-etcd

curl -L ${DOWNLOAD_URL}/${ETCD_VER}/etcd-${ETCD_VER}-linux-amd64.tar.gz -o /tmp/etcd-${ETCD_VER}-linux-amd64.tar.gz
tar xzvf /tmp/etcd-${ETCD_VER}-linux-amd64.tar.gz -C /tmp/test-etcd --strip-components=1

sudo cp /tmp/test-etcd/etcd* /data/etcd-install

/data/etcd-install/etcd --version
ETCDCTL_API=3 /data/etcd-install/etcdctl version
a)、101服务器上部署操作
# after transferring certs to remote machines
mkdir -p ${HOME}/certs
cp /data/etcd/certs/* ${HOME}/certs


# make sure etcd process has write access to this directory
# remove this directory if the cluster is new; keep if restarting etcd
# rm -rf /data/etcd/server


/data/etcd-install/etcd --name s1 \
  --data-dir /data/etcd/server \
  --listen-client-urls https://192.168.188.101:2379 \
  --advertise-client-urls https://192.168.188.101:2379 \
  --listen-peer-urls https://192.168.188.101:2380 \
  --initial-advertise-peer-urls https://192.168.188.101:2380 \
  --initial-cluster s1=https://192.168.188.101:2380,s2=https://192.168.188.102:2380,s3=https://192.168.188.103:2380 \
  --initial-cluster-token tkn \
  --initial-cluster-state new \
  --client-cert-auth \
  --trusted-ca-file ${HOME}/certs/etcd-root-ca.pem \
  --cert-file ${HOME}/certs/s1.pem \
  --key-file ${HOME}/certs/s1-key.pem \
  --peer-client-cert-auth \
  --peer-trusted-ca-file ${HOME}/certs/etcd-root-ca.pem \
  --peer-cert-file ${HOME}/certs/s1.pem \
  --peer-key-file ${HOME}/certs/s1-key.pem
b)、102服务器上部署操作
# after transferring certs to remote machines
mkdir -p ${HOME}/certs
cp /data/etcd/certs/* ${HOME}/certs


# make sure etcd process has write access to this directory
# remove this directory if the cluster is new; keep if restarting etcd
# rm -rf /data/etcd/server
/data/etcd-install/etcd --name s2 \
  --data-dir /data/etcd/server \
  --listen-client-urls https://192.168.188.102:2379 \
  --advertise-client-urls https://192.168.188.102:2379 \
  --listen-peer-urls https://192.168.188.102:2380 \
  --initial-advertise-peer-urls https://192.168.188.102:2380 \
  --initial-cluster s1=https://192.168.188.101:2380,s2=https://192.168.188.102:2380,s3=https://192.168.188.103:2380 \
  --initial-cluster-token tkn \
  --initial-cluster-state new \
  --client-cert-auth \
  --trusted-ca-file ${HOME}/certs/etcd-root-ca.pem \
  --cert-file ${HOME}/certs/s2.pem \
  --key-file ${HOME}/certs/s2-key.pem \
  --peer-client-cert-auth \
  --peer-trusted-ca-file ${HOME}/certs/etcd-root-ca.pem \
  --peer-cert-file ${HOME}/certs/s2.pem \
  --peer-key-file ${HOME}/certs/s2-key.pem
c)、103服务器上部署操作
# after transferring certs to remote machines
mkdir -p ${HOME}/certs
cp /data/etcd/certs/* ${HOME}/certs


# make sure etcd process has write access to this directory
# remove this directory if the cluster is new; keep if restarting etcd
# rm -rf /data/etcd/server


/data/etcd-install/etcd --name s3 \
  --data-dir /data/etcd/server \
  --listen-client-urls https://192.168.188.103:2379 \
  --advertise-client-urls https://192.168.188.103:2379 \
  --listen-peer-urls https://192.168.188.103:2380 \
  --initial-advertise-peer-urls https://192.168.188.103:2380 \
  --initial-cluster s1=https://192.168.188.101:2380,s2=https://192.168.188.102:2380,s3=https://192.168.188.103:2380 \
  --initial-cluster-token tkn \
  --initial-cluster-state new \
  --client-cert-auth \
  --trusted-ca-file ${HOME}/certs/etcd-root-ca.pem \
  --cert-file ${HOME}/certs/s3.pem \
  --key-file ${HOME}/certs/s3-key.pem \
  --peer-client-cert-auth \
  --peer-trusted-ca-file ${HOME}/certs/etcd-root-ca.pem \
  --peer-cert-file ${HOME}/certs/s3.pem \
  --peer-key-file ${HOME}/certs/s3-key.pem

三、验证ETCD集群是否正确安装

使用如下命令进行验证

[root@localhost ~]# ETCDCTL_API=3 /data/etcd-install/etcdctl   --endpoints 192.168.188.101:2379,192.168.188.102:2379,192.168.188.103:2379   --cacert ${HOME}/certs/etcd-root-ca.pem   --cert ${HOME}/certs/s1.pem   --key ${HOME}/certs/s1-key.pem   endpoint health
192.168.188.101:2379 is healthy: successfully committed proposal: took = 2.274473ms
192.168.188.103:2379 is healthy: successfully committed proposal: took = 2.150096ms
192.168.188.102:2379 is healthy: successfully committed proposal: took = 2.858808ms

windows 版本集群安装请参考:【ETCD】【实操篇】如何从源码编译并在window上搭建etcd集群?

更多推荐