免费资源网 – https://freexyz.cn/
目录一、Kubernetes 简介1.Kubernetes 架构设计图2.Kubernetes 常见组件介绍二、Kubernetes 二进制方式安装1.创建 CA 证书和密钥2.安装 ETCD 组件1）创建 ETCD 证书和密钥2）生成证书和密钥3）创建启动脚本4）启动 ETCD3.安装 Flannel 网络插件1）创建 Flannel 证书和密钥2）生成证书和密钥3）编写启动脚本4）启动并验证4.安装 Docker 服务1）创建启动脚本2）启动 Docker5.安装 Kubectl 服务1）创建 Admin 证书和密钥2）生成证书和密钥3）创建 Kubeconfig 文件4）创建 Kubectl 配置文件，并配置命令补全工具三、安装 Kubenetes 相关组件1.安装 Kube-APIServer 组件1）创建 Kubernetes 证书和密钥2）生成证书和密钥3）配置 Kube-APIServer 审计4）配置 Metrics-Server5）创建启动脚本6）启动 Kube-APIServer 并验证2.安装 Controller Manager 组件1）创建 Controller Manager 证书和密钥2）生成证书和密钥3）创建 Kubeconfig 文件4）创建启动脚本5）启动并验证3.安装 Kube-Scheduler 组件1）创建 Kube-Scheduler 证书和密钥2）生成证书和密钥3）创建 Kubeconfig 文件4）创建 Kube-Scheduler 配置文件5）创建启动脚本6）启动并验证4.安装 Kubelet 组件1）创建 Kubelet 启动脚本2）启动并验证3）Approve CSR 请求5）Kubelet API 接口配置5.安装 Kube-Proxy 组件1）创建 Kube-Proxy 证书和密钥2）生成证书和密钥3）创建 Kubeconfig 文件4）创建 Kube-Proxy 配置文件5）启动并验证6.安装 CoreDNS 插件1）修改 Coredns 配置2）创建 Coredns 并启动3）验证7.安装 Dashboard 仪表盘1）创建证书2）修改 Dashboard 配置3）验证
一、Kubernetes 简介
Kubernetes，也称为 K8s，是由 Google 公司开源的容器集群管理系统，在 Docker 技术的基础上，为容器化的应用提供部署运行、资源调度、服务发现和动态伸缩等一系列完整功能，提高了大规模容器集群管理的便捷性。Kubernetes 官方
1.Kubernetes 架构设计图
Kubernetes 是由一个 Master 和多个 Node 组成，Master 通过 API 提供服务，并接收 Kubectl 发送过来的请求来调度管理整个集群。
Kubectl 是 K8s 平台的管理命令。
2.Kubernetes 常见组件介绍
APIServer： 所有服务的统一访问入口，并提供认证、授权、访问控制、API 注册和发现等机制；
Controller Manager（控制器）： 主要就是用来维持 Pod 的一个副本数，比如故障检测、自动扩展、滚动更新等；
Scheduler（调度器）： 主要就是用来分配任务到合适的节点上（资源调度）
ETCD： 键值对数据库，存放了 K8s 集群中所有的重要信息（持久化）
Kubelet： 直接和容器引擎交互，用来维护容器的一个生命周期；同时也负责 Volume（CVI）和网络（CNI）的管理；
Kube-Porxy： 用于将规则写入至 iptables 或 IPVS 来实现服务的映射访问；
其它组件：
oreDNS：主要就是用来给 K8s 的 Service 提供一个域名和 IP 的对应解析关系。
Dashboard：主要就是用来给 K8s 提供一个 B/S 结构的访问体系（即，我们可以通过 Web 界面来对 K8s 进行管理）
Ingress Controller：主要就是用来实现 HTTP 代理（七层），官方的 Service 仅支持 TCPUDP 代理（四层）
Prometheus：主要就是用来给 K8s 提供一个监控能力，使我们能够更加清晰的看到 K8s 相关组件及 Pod 的使用情况。
ELK：主要就是用来给 K8s 提供一个日志分析平台。
Kubernetes 工作原理：
用户可以通过 Kubectl 命令来提交需要运行的 Docker Container 到 K8s 的 APIServer 组件中；
接着 APIServer 接收到用户提交的请求后，会将请求存储到 ETCD 这个键值对存储中；
然后由 Controller Manager 组件来创建出用户定义的控制器类型（Pod ReplicaSet Deployment DaemonSet 等）
然后 Scheduler 组件会对 ETCD 进行扫描，并将用户需要运行的 Docker Container 分配到合适的主机上；
最后由 Kubelet 组件来和 Docker 容器进行交互，创建、删除、停止容器等一系列操作。
kube-proxy 主要就是为 Service 提供服务的，来实现内部从 Pod 到 Service 和外部 NodePort 到 Service 的访问。
二、Kubernetes 二进制方式安装
我们下面的安装方式就是单纯的使用二进制方式安装，并没有对 Kube-APIServer 组件进行高可用配置，因为像我们安装 K8s 的话，其实主要还是为了学习 K8s，通过 K8s 来完成某些事情，所以并不需要关心高可用这块的东西。
要是对 Kubernetes 做高可用的话，其实并不难，像一些在云上的 K8s，一般都是通过 SLB 来代理到两台不同服务器上，来实现高可用；而像云下的 K8s，基本上也是如上，我们可以通过 Keepalived 加 Nginx 来实现高可用。
准备工作：
主机名操作系统IP 地址所需组件k8s-master01CentOS 7.4192.168.1.1所有组件都安装 （合理利用资源）k8s-master02CentOS 7.4192.168.1.2所有组件都安装k8s-nodeCentOS 7.4192.168.1.3docker kubelet kube-proxy1）在各个节点上配置主机名，并配置 Hosts 文件
[root@localhost ~]# hostnamectl set-hostname k8s-master01 [root@localhost ~]# bash [root@k8s-master01 ~]# cat <<END >> /etc/hosts 192.168.1.1 k8s-master01 192.168.1.2 k8s-master02 192.168.1.3 k8s-node01 END2）在 k8s-master01 上配置 SSH 密钥对，并将公钥发送给其余主机
[root@k8s-master01 ~]# ssh-keygen -t rsa # 三连回车 [root@k8s-master01 ~]# ssh-copy-id root@192.168.1.1 [root@k8s-master01 ~]# ssh-copy-id root@192.168.1.2 [root@k8s-master01 ~]# ssh-copy-id root@192.168.1.33）编写 K8s 初始环境脚本
[root@k8s-master01 ~]# vim k8s-init.sh #!/bin/bash #****************************************************************# # ScriptName: k8s-init.sh # Initialize the machine. This needs to be executed on every machine. # Mkdir k8s directory yum -y install wget ntpdate && ntpdate ntp1.aliyun.com wget -O /etc/yum.repos.d/CentOS-Base.repo https://mirrors.aliyun.com/repo/Centos-7.repo yum -y install epel-release mkdir -p /opt/k8s/bin/ mkdir -p /data/k8s/docker mkdir -p /data/k8s/k8s # Disable the SELinux. swapoff -a sed -i /swap/s/^/#/ /etc/fstab # Turn off and disable the firewalld. systemctl stop firewalld systemctl disable firewalld # Modify related kernel parameters & Disable the swap. cat > /etc/sysctl.d/k8s.conf << EOF net.ipv4.ip_forward = 1 net.bridge.bridge-nf-call-ip6tables = 1 net.bridge.bridge-nf-call-iptables = 1 net.ipv4.tcp_tw_recycle = 0 vm.swappiness = 0 vm.overcommit_memory = 1 vm.panic_on_oom = 0 net.ipv6.conf.all.disable_ipv6 = 1 EOF sysctl -p /etc/sysctl.d/k8s.conf >& /dev/null # Add ipvs modules cat > /etc/sysconfig/modules/ipvs.modules << EOF #!/bin/bash modprobe — ip_vs modprobe — ip_vs_rr modprobe — ip_vs_wrr modprobe — ip_vs_sh modprobe — br_netfilter modprobe — nf_conntrack modprobe — nf_conntrack_ipv4 EOF chmod 755 /etc/sysconfig/modules/ipvs.modules source /etc/sysconfig/modules/ipvs.modules # Install rpm yum install -y conntrack ntpdate ntp ipvsadm ipset jq iptables curl sysstat libseccomp wget gcc gcc-c++ make libnl libnl-devel libnfnetlink-devel openssl-devel vim openssl-devel bash-completion # ADD k8s bin to PATH echo export PATH=/opt/k8s/bin:$PATH >> /root/.bashrc && chmod +x /root/.bashrc && source /root/.bashrc [root@k8s-master01 ~]# bash k8s-init.sh4）配置环境变量
[root@k8s-master01 ~]# vim environment.sh #!/bin/bash # 生成 EncryptionConfig 所需的加密 Key export ENCRYPTION_KEY=$(head -c 32 /dev/urandom | base64) # 集群 Master 机器 IP 数组 export MASTER_IPS=(192.168.1.1 192.168.1.2) # 集群 Master IP 对应的主机名数组 export MASTER_NAMES=(k8s-master01 k8s-master02) # 集群 Node 机器 IP 数组 export NODE_IPS=(192.168.1.3) # 集群 Node IP 对应的主机名数组 export NODE_NAMES=(k8s-node01) # 集群所有机器 IP 数组 export ALL_IPS=(192.168.1.1 192.168.1.2 192.168.1.3) # 集群所有 IP 对应的主机名数组 export ALL_NAMES=(k8s-master01 k8s-master02 k8s-node01) # Etcd 集群服务地址列表 export ETCD_ENDPOINTS=”https://192.168.1.1:2379,https://192.168.1.2:2379″ # Etcd 集群间通信的 IP 和端口 export ETCD_NODES=”k8s-master01=https://192.168.1.1:2380,k8s-master02=https://192.168.1.2:2380″ # Kube-apiserver 的 IP 和端口 export KUBE_APISERVER=”https://192.168.1.1:6443″ # 节点间互联网络接口名称 export IFACE=”ens32″ # Etcd 数据目录 export ETCD_DATA_DIR=”/data/k8s/etcd/data” # Etcd WAL 目录. 建议是 SSD 磁盘分区. 或者和 ETCD_DATA_DIR 不同的磁盘分区 export ETCD_WAL_DIR=”/data/k8s/etcd/wal” # K8s 各组件数据目录 export K8S_DIR=”/data/k8s/k8s” # Docker 数据目录 export DOCKER_DIR=”/data/k8s/docker” ## 以下参数一般不需要修改 # TLS Bootstrapping 使用的 Token. 可以使用命令 head -c 16 /dev/urandom | od -An -t x | tr -d 生成 BOOTSTRAP_TOKEN=”41f7e4ba8b7be874fcff18bf5cf41a7c” # 最好使用当前未用的网段来定义服务网段和 Pod 网段 # 服务网段. 部署前路由不可达. 部署后集群内路由可达(kube-proxy 保证) SERVICE_CIDR=”10.20.0.0/16″ # Pod 网段. 建议 /16 段地址. 部署前路由不可达. 部署后集群内路由可达(flanneld 保证) CLUSTER_CIDR=”10.10.0.0/16″ # 服务端口范围 (NodePort Range) export NODE_PORT_RANGE=”1-65535″ # Flanneld 网络配置前缀 export FLANNEL_ETCD_PREFIX=”/kubernetes/network” # Kubernetes 服务 IP (一般是 SERVICE_CIDR 中第一个 IP) export CLUSTER_KUBERNETES_SVC_IP=”10.20.0.1″ # 集群 DNS 服务 IP (从 SERVICE_CIDR 中预分配) export CLUSTER_DNS_SVC_IP=”10.20.0.254″ # 集群 DNS 域名(末尾不带点号) export CLUSTER_DNS_DOMAIN=”cluster.local” # 将二进制目录 /opt/k8s/bin 加到 PATH 中 export PATH=/opt/k8s/bin:$PATH上面像那些 IP 地址和网卡啥的，你们要改成自身对应的信息。
[root@k8s-master01 ~]# chmod +x environment.sh && source environment.sh下面的这些操作，我们只需要在 k8s-master01 主机上操作即可（因为下面我们会通过 for 循环来发送到其余主机上）
1.创建 CA 证书和密钥
因为 Kubernetes 系统的各个组件需要使用 TLS 证书对其通信加密以及授权认证，所以我们需要在安装前先生成相关的 TLS 证书；我们可以使用 openssl cfssl easyrsa 来生成 Kubernetes 的相关证书，我们下面使用的是 cfssl 方式。
1）安装 cfssl 工具集
[root@k8s-master01 ~]# mkdir -p /opt/k8s/cert [root@k8s-master01 ~]# curl -L https://pkg.cfssl.org/R1.2/cfssl_linux-amd64 -o /opt/k8s/bin/cfssl [root@k8s-master01 ~]# curl -L https://pkg.cfssl.org/R1.2/cfssljson_linux-amd64 -o /opt/k8s/bin/cfssljson [root@k8s-master01 ~]# curl -L https://pkg.cfssl.org/R1.2/cfssl-certinfo_linux-amd64 -o /opt/k8s/bin/cfssl-certinfo [root@k8s-master01 ~]# chmod +x /opt/k8s/bin/*2）创建根证书配置文件
[root@k8s-master01 ~]# mkdir -p /opt/k8s/work [root@k8s-master01 ~]# cd /opt/k8s/work/ [root@k8s-master01 work]# cat > ca-config.json << EOF { “signing”: { “default”: { “expiry”: “876000h” }, “profiles”: { “kubernetes”: { “expiry”: “876000h”, “usages”: [ “signing”, “key encipherment”, “server auth”, “client auth” ] } } } } EOFsigning：表示当前证书可用于签名其它证书；
server auth：表示 Client 可以用这个 CA 对 Server 提供的证书进行校验；
client auth：表示 Server 可以用这个 CA 对 Client 提供的证书进行验证；
“expiry”: “876000h”：表示当前证书有效期为 100 年；
3）创建根证书签名请求文件
[root@k8s-master01 work]# cat > ca-csr.json << EOF { “CN”: “kubernetes”, “key”: { “algo”: “rsa”, “size”: 2048 }, “names”: [ { “C”: “CN”, “ST”: “Shanghai”, “L”: “Shanghai”, “O”: “k8s”, “OU”: “System” } ], “ca”: { “expiry”: “876000h” } } EOFCN：Kube-APIServer 将会把这个字段作为请求的用户名，来让浏览器验证网站是否合法。
C：国家；ST：州，省；L：地区，城市；O：组织名称，公司名称；OU：组织单位名称，公司部门。
4）生成 CA 密钥 ca-key.pem 和证书 ca.pem
[root@k8s-master01 work]# cfssl gencert -initca ca-csr.json | cfssljson -bare ca生成证书后，因为 Kubernetes 集群需要 双向 TLS 认证，所以我们可以将生成的文件传送到所有主机中。
5）使用 for 循环来遍历数组，将配置发送给所有主机
[root@k8s-master01 work]# for all_ip in ${ALL_IPS[@]} do echo “>>> ${all_ip}” ssh root@${all_ip} “mkdir -p /etc/kubernetes/cert” scp ca*.pem ca-config.json root@${all_ip}:/etc/kubernetes/cert done2.安装 ETCD 组件
ETCD 是基于 Raft 的分布式 key-value 存储系统，由 CoreOS 开发，常用于服务发现、共享配置以及并发控制（如 leader 选举、分布式锁等）；Kubernetes 主要就是用 ETCD 来存储所有的运行数据。
下载 ETCD
[root@k8s-master01 work]# wget https://github.com/etcd-io/etcd/releases/download/v3.3.22/etcd-v3.3.22-linux-amd64.tar.gz [root@k8s-master01 work]# tar -zxf etcd-v3.3.22-linux-amd64.tar.gz [root@k8s-master01 work]# for master_ip in ${MASTER_IPS[@]} do echo “>>> ${master_ip}” scp etcd-v3.3.22-linux-amd64/etcd* root@${master_ip}:/opt/k8s/bin ssh root@${master_ip} “chmod +x /opt/k8s/bin/*” done 1）创建 ETCD 证书和密钥 [root@k8s-master01 work]# cat > etcd-csr.json << EOF { “CN”: “etcd”, “hosts”: [ “127.0.0.1”, “192.168.1.1”, “192.168.1.2” ], “key”: { “algo”: “rsa”, “size”: 2048 }, “names”: [ { “C”: “CN”, “ST”: “Shanghai”, “L”: “Shanghai”, “O”: “k8s”, “OU”: “System” } ] } EOFhosts：用来指定给 ETCD 授权的 IP 地址或域名列表。
2）生成证书和密钥 [root@k8s-master01 work]# cfssl gencert -ca=/opt/k8s/work/ca.pem -ca-key=/opt/k8s/work/ca-key.pem -config=/opt/k8s/work/ca-config.json -profile=kubernetes etcd-csr.json | cfssljson -bare etcd [root@k8s-master01 work]# for master_ip in ${MASTER_IPS[@]} do echo “>>> ${master_ip}” ssh root@${master_ip} “mkdir -p /etc/etcd/cert” scp etcd*.pem root@${master_ip}:/etc/etcd/cert/ done 3）创建启动脚本 [root@k8s-master01 work]# cat > etcd.service.template << EOF [Unit] Description=Etcd Server After=network.target After=network-online.target Wants=network-online.target Documentation=https://github.com/coreos [Service] Type=notify WorkingDirectory=${ETCD_DATA_DIR} ExecStart=/opt/k8s/bin/etcd –enable-v2=true –data-dir=${ETCD_DATA_DIR} –wal-dir=${ETCD_WAL_DIR} –name=##MASTER_NAME## –cert-file=/etc/etcd/cert/etcd.pem –key-file=/etc/etcd/cert/etcd-key.pem –trusted-ca-file=/etc/kubernetes/cert/ca.pem –peer-cert-file=/etc/etcd/cert/etcd.pem –peer-key-file=/etc/etcd/cert/etcd-key.pem –peer-trusted-ca-file=/etc/kubernetes/cert/ca.pem –peer-client-cert-auth –client-cert-auth –listen-peer-urls=https://##MASTER_IP##:2380 –initial-advertise-peer-urls=https://##MASTER_IP##:2380 –listen-client-urls=https://##MASTER_IP##:2379,http://127.0.0.1:2379 –advertise-client-urls=https://##MASTER_IP##:2379 –initial-cluster-token=etcd-cluster-0 –initial-cluster=${ETCD_NODES} –initial-cluster-state=new –auto-compaction-mode=periodic –auto-compaction-retention=1 –max-request-bytes=33554432 –quota-backend-bytes=6442450944 –heartbeat-interval=250 –election-timeout=2000 Restart=on-failure RestartSec=5 LimitNOFILE=65536 [Install] WantedBy=multi-user.target EOF [root@k8s-master01 work]# for (( A=0; A < 2; A++ )) do sed -e “s/##MASTER_NAME##/${MASTER_NAMES[A]}/” -e “s/##MASTER_IP##/${MASTER_IPS[A]}/” etcd.service.template > etcd-${MASTER_IPS[A]}.service done 4）启动 ETCD [root@k8s-master01 work]# for master_ip in ${MASTER_IPS[@]} do echo “>>> ${master_ip}” scp etcd-${master_ip}.service root@${master_ip}:/etc/systemd/system/etcd.service ssh root@${master_ip} “mkdir -p ${ETCD_DATA_DIR} ${ETCD_WAL_DIR}” ssh root@${master_ip} “systemctl daemon-reload && systemctl enable etcd && systemctl restart etcd” done查看 ETCD 当前的 Leader（领导）
[root@k8s-master01 work]# ETCDCTL_API=3 /opt/k8s/bin/etcdctl -w table –cacert=/etc/kubernetes/cert/ca.pem –cert=/etc/etcd/cert/etcd.pem –key=/etc/etcd/cert/etcd-key.pem –endpoints=${ETCD_ENDPOINTS} endpoint status
3.安装 Flannel 网络插件
Flannel 是一种基于 overlay 网络的跨主机容器网络解决方案，也就是将 TCP 数据封装在另一种网络包里面进行路由转发和通信。Flannel 是使用 Go 语言开发的，主要就是用来让不同主机内的容器实现互联。
下载 Flannel
[root@k8s-master01 work]# mkdir flannel [root@k8s-master01 work]# wget https://github.com/coreos/flannel/releases/download/v0.11.0/flannel-v0.11.0-linux-amd64.tar.gz [root@k8s-master01 work]# tar -zxf flannel-v0.11.0-linux-amd64.tar.gz -C flannel [root@k8s-master01 work]# for all_ip in ${ALL_IPS[@]} do echo “>>> ${all_ip}” scp flannel/{flanneld,mk-docker-opts.sh} root@${all_ip}:/opt/k8s/bin/ ssh root@${all_ip} “chmod +x /opt/k8s/bin/*” done 1）创建 Flannel 证书和密钥 [root@k8s-master01 work]# cat > flanneld-csr.json << EOF { “CN”: “flanneld”, “hosts”: [], “key”: { “algo”: “rsa”, “size”: 2048 }, “names”: [ { “C”: “CN”, “ST”: “Shanghai”, “L”: “Shanghai”, “O”: “k8s”, “OU”: “System” } ] } EOF 2）生成证书和密钥 [root@k8s-master01 work]# cfssl gencert -ca=/opt/k8s/work/ca.pem -ca-key=/opt/k8s/work/ca-key.pem -config=/opt/k8s/work/ca-config.json -profile=kubernetes flanneld-csr.json | cfssljson -bare flanneld [root@k8s-master01 work]# for all_ip in ${ALL_IPS[@]} do echo “>>> ${all_ip}” ssh root@${all_ip} “mkdir -p /etc/flanneld/cert” scp flanneld*.pem root@${all_ip}:/etc/flanneld/cert done配置 Pod 的网段信息
[root@k8s-master01 work]# etcdctl –endpoints=${ETCD_ENDPOINTS} –ca-file=/opt/k8s/work/ca.pem –cert-file=/opt/k8s/work/flanneld.pem –key-file=/opt/k8s/work/flanneld-key.pem mk ${FLANNEL_ETCD_PREFIX}/config {“Network”:”${CLUSTER_CIDR}”, “SubnetLen”: 21, “Backend”: {“Type”: “vxlan”}}
1）查看 Pod 网段信息
[root@k8s-master01 work]# etcdctl –endpoints=${ETCD_ENDPOINTS} –ca-file=/etc/kubernetes/cert/ca.pem –cert-file=/etc/flanneld/cert/flanneld.pem –key-file=/etc/flanneld/cert/flanneld-key.pem get ${FLANNEL_ETCD_PREFIX}/config
2）查看已分配的 Pod 子网段列表
[root@k8s-master01 work]# etcdctl –endpoints=${ETCD_ENDPOINTS} –ca-file=/etc/kubernetes/cert/ca.pem –cert-file=/etc/flanneld/cert/flanneld.pem –key-file=/etc/flanneld/cert/flanneld-key.pem ls ${FLANNEL_ETCD_PREFIX}/subnets
3）查看某一 Pod 网段对应的节点 IP 和 Flannel 接口地址
[root@k8s-master01 work]# etcdctl –endpoints=${ETCD_ENDPOINTS} –ca-file=/etc/kubernetes/cert/ca.pem –cert-file=/etc/flanneld/cert/flanneld.pem –key-file=/etc/flanneld/cert/flanneld-key.pem get ${FLANNEL_ETCD_PREFIX}/subnets/10.10.208.0-21
4.安装 Docker 服务
Docker 运行和管理容器，Kubelet 通过 Container Runtime Interface (CRI) 与它进行交互。
下载 Docker
[root@k8s-master01 work]# wget https://download.docker.com/linux/static/stable/x86_64/docker-19.03.12.tgz [root@k8s-master01 work]# tar -zxf docker-19.03.12.tgz安装 Docker
[root@k8s-master01 work]# for all_ip in ${ALL_IPS[@]} do echo “>>> ${all_ip}” scp docker/* root@${all_ip}:/opt/k8s/bin/ ssh root@${all_ip} “chmod +x /opt/k8s/bin/*” done 1）创建启动脚本 [root@k8s-master01 work]# cat > docker.service << “EOF” [Unit] Description=Docker Application Container Engine Documentation=http://docs.docker.io [Service] WorkingDirectory=##DOCKER_DIR## Environment=”PATH=/opt/k8s/bin:/bin:/sbin:/usr/bin:/usr/sbin” EnvironmentFile=-/run/flannel/docker ExecStart=/opt/k8s/bin/dockerd $DOCKER_NETWORK_OPTIONS ExecReload=/bin/kill -s HUP $MAINPID Restart=on-failure RestartSec=5 LimitNOFILE=infinity LimitNPROC=infinity LimitCORE=infinity Delegate=yes KillMode=process [Install] WantedBy=multi-user.target EOF [root@k8s-master01 work]# sed -i -e “s|##DOCKER_DIR##|${DOCKER_DIR}|” docker.service [root@k8s-master01 work]# for all_ip in ${ALL_IPS[@]} do echo “>>> ${all_ip}” scp docker.service root@${all_ip}:/etc/systemd/system/ done配置 daemon.json 文件
[root@k8s-master01 work]# cat > daemon.json << EOF { “registry-mirrors”: [“https://ipbtg5l0.mirror.aliyuncs.com”], “exec-opts”: [“native.cgroupdriver=cgroupfs”], “data-root”: “${DOCKER_DIR}/data”, “exec-root”: “${DOCKER_DIR}/exec”, “log-driver”: “json-file”, “log-opts”: { “max-size”: “100m”, “max-file”: “5” }, “storage-driver”: “overlay2”, “storage-opts”: [ “overlay2.override_kernel_check=true” ] } EOF [root@k8s-master01 work]# cat > daemon.json << EOF { “registry-mirrors”: [“https://ipbtg5l0.mirror.aliyuncs.com”], “exec-opts”: [“native.cgroupdriver=cgroupfs”], “data-root”: “${DOCKER_DIR}/data”, “exec-root”: “${DOCKER_DIR}/exec”, “log-driver”: “json-file”, “log-opts”: { “max-size”: “100m”, “max-file”: “5” }, “storage-driver”: “overlay2”, “storage-opts”: [ “overlay2.override_kernel_check=true” ] } EOF 2）启动 Docker [root@k8s-master01 work]# for all_ip in ${ALL_IPS[@]} do echo “>>> ${all_ip}” ssh root@${all_ip} “systemctl daemon-reload && systemctl enable docker –now” done5.安装 Kubectl 服务
下载 Kubectl
[root@k8s-master01 work]# wget https://storage.googleapis.com/kubernetes-release/release/v1.18.3/kubernetes-client-linux-amd64.tar.gz [root@k8s-master01 work]# tar -zxf kubernetes-client-linux-amd64.tar.gz [root@k8s-master01 work]# for master_ip in ${MASTER_IPS[@]} do echo “>>> ${master_ip}” scp kubernetes/client/bin/kubectl root@${master_ip}:/opt/k8s/bin/ ssh root@${master_ip} “chmod +x /opt/k8s/bin/*” done 1）创建 Admin 证书和密钥 [root@k8s-master01 work]# cat > admin-csr.json << EOF { “CN”: “admin”, “hosts”: [], “key”: { “algo”: “rsa”, “size”: 2048 }, “names”: [ { “C”: “CN”, “ST”: “Shanghai”, “L”: “Shanghai”, “O”: “system:masters”, “OU”: “System” } ] } EOF 2）生成证书和密钥 [root@k8s-master01 work]# cfssl gencert -ca=/opt/k8s/work/ca.pem -ca-key=/opt/k8s/work/ca-key.pem -config=/opt/k8s/work/ca-config.json -profile=kubernetes admin-csr.json | cfssljson -bare admin 3）创建 Kubeconfig 文件配置集群参数
[root@k8s-master01 work]# kubectl config set-cluster kubernetes –certificate-authority=/opt/k8s/work/ca.pem –embed-certs=true –server=${KUBE_APISERVER} –kubeconfig=kubectl.kubeconfig配置客户端认证参数
[root@k8s-master01 work]# kubectl config set-credentials admin –client-certificate=/opt/k8s/work/admin.pem –client-key=/opt/k8s/work/admin-key.pem –embed-certs=true –kubeconfig=kubectl.kubeconfig配置上下文参数
[root@k8s-master01 work]# kubectl config set-context kubernetes –cluster=kubernetes –user=admin –kubeconfig=kubectl.kubeconfig配置默认上下文
[root@k8s-master01 work]# kubectl config use-context kubernetes –kubeconfig=kubectl.kubeconfig 4）创建 Kubectl 配置文件，并配置命令补全工具 [root@k8s-master01 work]# for master_ip in ${MASTER_IPS[@]} do echo “>>> ${master_ip}” ssh root@${master_ip} “mkdir -p ~/.kube” scp kubectl.kubeconfig root@${master_ip}:~/.kube/config ssh root@${master_ip} “echo export KUBECONFIG=$HOME/.kube/config >> ~/.bashrc” ssh root@${master_ip} “echo source <(kubectl completion bash) >> ~/.bashrc” done下面命令需要在 k8s-master01 和 k8s-master02 上配置：
[root@k8s-master01 work]# source /usr/share/bash-completion/bash_completion [root@k8s-master01 work]# source <(kubectl completion bash) [root@k8s-master01 work]# bash ~/.bashrc三、安装 Kubenetes 相关组件
1.安装 Kube-APIServer 组件
下载 Kubernetes 二进制文件
[root@k8s-master01 work]# wget https://storage.googleapis.com/kubernetes-release/release/v1.18.3/kubernetes-server-linux-amd64.tar.gz [root@k8s-master01 work]# tar -zxf kubernetes-server-linux-amd64.tar.gz [root@k8s-master01 work]# cd kubernetes [root@k8s-master01 kubernetes]# tar -zxf kubernetes-src.tar.gz [root@k8s-master01 kubernetes]# cd .. [root@k8s-master01 work]# for master_ip in ${MASTER_IPS[@]} do echo “>>> ${master_ip}” scp -rp kubernetes/server/bin/{apiextensions-apiserver,kube-apiserver,kube-controller-manager,kube-scheduler,kubeadm,kubectl,mounter} root@${master_ip}:/opt/k8s/bin/ ssh root@${master_ip} “chmod +x /opt/k8s/bin/*” done 1）创建 Kubernetes 证书和密钥 [root@k8s-master01 work]# cat > kubernetes-csr.json << EOF { “CN”: “kubernetes”, “hosts”: [ “127.0.0.1”, “192.168.1.1”, “192.168.1.2”, “${CLUSTER_KUBERNETES_SVC_IP}”, “kubernetes”, “kubernetes.default”, “kubernetes.default.svc”, “kubernetes.default.svc.cluster”, “kubernetes.default.svc.cluster.local.” ], “key”: { “algo”: “rsa”, “size”: 2048 }, “names”: [ { “C”: “CN”, “ST”: “Shanghai”, “L”: “Shanghai”, “O”: “k8s”, “OU”: “System” } ] } EOF 2）生成证书和密钥 [root@k8s-master01 work]# cfssl gencert -ca=/opt/k8s/work/ca.pem -ca-key=/opt/k8s/work/ca-key.pem -config=/opt/k8s/work/ca-config.json -profile=kubernetes kubernetes-csr.json | cfssljson -bare kubernetes [root@k8s-master01 work]# for master_ip in ${MASTER_IPS[@]} do echo “>>> ${master_ip}” ssh root@${master_ip} “mkdir -p /etc/kubernetes/cert” scp kubernetes*.pem root@${master_ip}:/etc/kubernetes/cert/ done 3）配置 Kube-APIServer 审计创建加密配置文件
[root@k8s-master01 work]# cat > encryption-config.yaml << EOF kind: EncryptionConfig apiVersion: v1 resources: – resources: – secrets providers: – aescbc: keys: – name: zhangsan secret: ${ENCRYPTION_KEY} – identity: {} EOF [root@k8s-master01 work]# for master_ip in ${MASTER_IPS[@]} do echo “>>> ${master_ip}” scp encryption-config.yaml root@${master_ip}:/etc/kubernetes/encryption-config.yaml done创建审计策略文件
[root@k8s-master01 work]# cat > audit-policy.yaml << EOF apiVersion: audit.k8s.io/v1beta1 kind: Policy rules: # The following requests were manually identified as high-volume and low-risk, so drop them. – level: None resources: – group: “” resources: – endpoints – services – services/status users: – system:kube-proxy verbs: – watch – level: None resources: – group: “” resources: – nodes – nodes/status userGroups: – system:nodes verbs: – get – level: None namespaces: – kube-system resources: – group: “” resources: – endpoints users: – system:kube-controller-manager – system:kube-scheduler – system:serviceaccount:kube-system:endpoint-controller verbs: – get – update – level: None resources: – group: “” resources: – namespaces – namespaces/status – namespaces/finalize users: – system:apiserver verbs: – get # Dont log HPA fetching metrics. – level: None resources: – group: metrics.k8s.io users: – system:kube-controller-manager verbs: – get – list # Dont log these read-only URLs. – level: None nonResourceURLs: – /healthz* – /version – /swagger* # Dont log events requests. – level: None resources: – group: “” resources: – events # node and pod status calls from nodes are high-volume and can be large, dont log responses for expected updates from nodes – level: Request omitStages: – RequestReceived resources: – group: “” resources: – nodes/status – pods/status users: – kubelet – system:node-problem-detector – system:serviceaccount:kube-system:node-problem-detector verbs: – update – patch – level: Request omitStages: – RequestReceived resources: – group: “” resources: – nodes/status – pods/status userGroups: – system:nodes verbs: – update – patch # deletecollection calls can be large, dont log responses for expected namespace deletions – level: Request omitStages: – RequestReceived users: – system:serviceaccount:kube-system:namespace-controller verbs: – deletecollection # Secrets, ConfigMaps, and TokenReviews can contain sensitive & binary data, # so only log at the Metadata level. – level: Metadata omitStages: – RequestReceived resources: – group: “” resources: – secrets – configmaps – group: authentication.k8s.io resources: – tokenreviews # Get repsonses can be large; skip them. – level: Request omitStages: – RequestReceived resources: – group: “” – group: admissionregistration.k8s.io – group: apiextensions.k8s.io – group: apiregistration.k8s.io – group: apps – group: authentication.k8s.io – group: authorization.k8s.io – group: autoscaling – group: batch – group: certificates.k8s.io – group: extensions – group: metrics.k8s.io – group: networking.k8s.io – group: policy – group: rbac.authorization.k8s.io – group: scheduling.k8s.io – group: settings.k8s.io – group: storage.k8s.io verbs: – get – list – watch # Default level for known APIs – level: RequestResponse omitStages: – RequestReceived resources: – group: “” – group: admissionregistration.k8s.io – group: apiextensions.k8s.io – group: apiregistration.k8s.io – group: apps – group: authentication.k8s.io – group: authorization.k8s.io – group: autoscaling – group: batch – group: certificates.k8s.io – group: extensions – group: metrics.k8s.io – group: networking.k8s.io – group: policy – group: rbac.authorization.k8s.io – group: scheduling.k8s.io – group: settings.k8s.io – group: storage.k8s.io # Default level for all other requests. – level: Metadata omitStages: – RequestReceived EOF [root@k8s-master01 work]# for master_ip in ${MASTER_IPS[@]} do echo “>>> ${master_ip}” scp audit-policy.yaml root@${master_ip}:/etc/kubernetes/audit-policy.yaml done 4）配置 Metrics-Server创建 metrics-server 的 CA 证书请求文件
[root@k8s-master01 work]# cat > proxy-client-csr.json << EOF { “CN”: “system:metrics-server”, “hosts”: [], “key”: { “algo”: “rsa”, “size”: 2048 }, “names”: [ { “C”: “CN”, “ST”: “Shanghai”, “L”: “Shanghai”, “O”: “k8s”, “OU”: “System” } ] } EOF生成证书和密钥
[root@k8s-master01 work]# cfssl gencert -ca=/opt/k8s/work/ca.pem -ca-key=/opt/k8s/work/ca-key.pem -config=/opt/k8s/work/ca-config.json -profile=kubernetes proxy-client-csr.json | cfssljson -bare proxy-client [root@k8s-master01 work]# for master_ip in ${MASTER_IPS[@]} do echo “>>> ${master_ip}” scp proxy-client*.pem root@${master_ip}:/etc/kubernetes/cert/ done 5）创建启动脚本 [root@k8s-master01 work]# cat > kube-apiserver.service.template << EOF [Unit] Description=Kubernetes API Server Documentation=https://github.com/GoogleCloudPlatform/kubernetes After=network.target [Service] WorkingDirectory=${K8S_DIR}/kube-apiserver ExecStart=/opt/k8s/bin/kube-apiserver –insecure-port=0 –secure-port=6443 –bind-address=##MASTER_IP## –advertise-address=##MASTER_IP## –default-not-ready-toleration-seconds=360 –default-unreachable-toleration-seconds=360 –feature-gates=DynamicAuditing=true –max-mutating-requests-inflight=2000 –max-requests-inflight=4000 –default-watch-cache-size=200 –delete-collection-workers=2 –encryption-provider-config=/etc/kubernetes/encryption-config.yaml –etcd-cafile=/etc/kubernetes/cert/ca.pem –etcd-certfile=/etc/kubernetes/cert/kubernetes.pem –etcd-keyfile=/etc/kubernetes/cert/kubernetes-key.pem –etcd-servers=${ETCD_ENDPOINTS} –tls-cert-file=/etc/kubernetes/cert/kubernetes.pem –tls-private-key-file=/etc/kubernetes/cert/kubernetes-key.pem –audit-dynamic-configuration –audit-log-maxage=30 –audit-log-maxbackup=3 –audit-log-maxsize=100 –audit-log-truncate-enabled=true –audit-log-path=${K8S_DIR}/kube-apiserver/audit.log –audit-policy-file=/etc/kubernetes/audit-policy.yaml –profiling –anonymous-auth=false –client-ca-file=/etc/kubernetes/cert/ca.pem –enable-bootstrap-token-auth=true –requestheader-allowed-names=”system:metrics-server” –requestheader-client-ca-file=/etc/kubernetes/cert/ca.pem –requestheader-extra-headers-prefix=X-Remote-Extra- –requestheader-group-headers=X-Remote-Group –requestheader-username-headers=X-Remote-User –service-account-key-file=/etc/kubernetes/cert/ca.pem –authorization-mode=Node,RBAC –runtime-config=api/all=true –enable-admission-plugins=NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultStorageClass,DefaultTolerationSeconds,MutatingAdmissionWebhook,ValidatingAdmissionWebhook,ResourceQuota,NodeRestriction –allow-privileged=true –apiserver-count=3 –event-ttl=168h –kubelet-certificate-authority=/etc/kubernetes/cert/ca.pem –kubelet-client-certificate=/etc/kubernetes/cert/kubernetes.pem –kubelet-client-key=/etc/kubernetes/cert/kubernetes-key.pem –kubelet-https=true –kubelet-timeout=10s –proxy-client-cert-file=/etc/kubernetes/cert/proxy-client.pem –proxy-client-key-file=/etc/kubernetes/cert/proxy-client-key.pem –service-cluster-ip-range=${SERVICE_CIDR} –service-node-port-range=${NODE_PORT_RANGE} –logtostderr=true –v=2 Restart=on-failure RestartSec=10 Type=notify LimitNOFILE=65535 [Install] WantedBy=multi-user.target EOF 6）启动 Kube-APIServer 并验证 [root@k8s-master01 work]# for (( A=0; A < 2; A++ )) do sed -e “s/##MASTER_NAME##/${MASTER_NAMES[A]}/” -e “s/##MASTER_IP##/${MASTER_IPS[A]}/” kube-apiserver.service.template > kube-apiserver-${MASTER_IPS[A]}.service done [root@k8s-master01 work]# for master_ip in ${MASTER_IPS[@]} do echo “>>> ${master_ip}” scp kube-apiserver-${master_ip}.service root@${master_ip}:/etc/systemd/system/kube-apiserver.service ssh root@${master_ip} “mkdir -p ${K8S_DIR}/kube-apiserver” ssh root@${master_ip} “systemctl daemon-reload && systemctl enable kube-apiserver –now” done查看 Kube-APIServer 写入 ETCD 的数据
[root@k8s-master01 work]# ETCDCTL_API=3 etcdctl –endpoints=${ETCD_ENDPOINTS} –cacert=/opt/k8s/work/ca.pem –cert=/opt/k8s/work/etcd.pem –key=/opt/k8s/work/etcd-key.pem get /registry/ –prefix –keys-only查看集群信息
[root@k8s-master01 work]# kubectl cluster-info [root@k8s-master01 work]# kubectl get all –all-namespaces [root@k8s-master01 work]# kubectl get componentstatuses [root@k8s-master01 work]# netstat -anpt | grep 6443
授予 kube-apiserver 访问 kubelet API 的权限
[root@k8s-master01 work]# kubectl create clusterrolebinding kube-apiserver:kubelet-apis –clusterrole=system:kubelet-api-admin –user kubernetes2.安装 Controller Manager 组件
1）创建 Controller Manager 证书和密钥 [root@k8s-master01 work]# cat > kube-controller-manager-csr.json << EOF { “CN”: “system:kube-controller-manager”, “hosts”: [ “127.0.0.1”, “192.168.1.1”, “192.168.1.2” ], “key”: { “algo”: “rsa”, “size”: 2048 }, “names”: [ { “C”: “CN”, “ST”: “Shanghai”, “L”: “Shanghai”, “O”: “system:kube-controller-manager”, “OU”: “System” } ] } EOF 2）生成证书和密钥 [root@k8s-master01 work]# cfssl gencert -ca=/opt/k8s/work/ca.pem -ca-key=/opt/k8s/work/ca-key.pem -config=/opt/k8s/work/ca-config.json -profile=kubernetes kube-controller-manager-csr.json | cfssljson -bare kube-controller-manager [root@k8s-master01 work]# for master_ip in ${MASTER_IPS[@]} do echo “>>> ${master_ip}” scp kube-controller-manager*.pem root@${master_ip}:/etc/kubernetes/cert/ done 3）创建 Kubeconfig 文件 [root@k8s-master01 work]# kubectl config set-cluster kubernetes –certificate-authority=/opt/k8s/work/ca.pem –embed-certs=true –server=${KUBE_APISERVER} –kubeconfig=kube-controller-manager.kubeconfig [root@k8s-master01 work]# kubectl config set-credentials system:kube-controller-manager –client-certificate=kube-controller-manager.pem –client-key=kube-controller-manager-key.pem –embed-certs=true –kubeconfig=kube-controller-manager.kubeconfig [root@k8s-master01 work]# kubectl config set-context system:kube-controller-manager –cluster=kubernetes –user=system:kube-controller-manager –kubeconfig=kube-controller-manager.kubeconfig [root@k8s-master01 work]# kubectl config use-context system:kube-controller-manager –kubeconfig=kube-controller-manager.kubeconfig [root@k8s-master01 work]# for master_ip in ${MASTER_IPS[@]} do echo “>>> ${master_ip}” scp kube-controller-manager.kubeconfig root@${master_ip}:/etc/kubernetes/ done 4）创建启动脚本 [root@k8s-master01 work]# cat > kube-controller-manager.service.template << EOF [Unit] Description=Kubernetes Controller Manager Documentation=https://github.com/GoogleCloudPlatform/kubernetes [Service] WorkingDirectory=${K8S_DIR}/kube-controller-manager ExecStart=/opt/k8s/bin/kube-controller-manager –secure-port=10257 –bind-address=127.0.0.1 –profiling –cluster-name=kubernetes –controllers=*,bootstrapsigner,tokencleaner –kube-api-qps=1000 –kube-api-burst=2000 –leader-elect –use-service-account-credentials –concurrent-service-syncs=2 –tls-cert-file=/etc/kubernetes/cert/kube-controller-manager.pem –tls-private-key-file=/etc/kubernetes/cert/kube-controller-manager-key.pem –authentication-kubeconfig=/etc/kubernetes/kube-controller-manager.kubeconfig –client-ca-file=/etc/kubernetes/cert/ca.pem –requestheader-allowed-names=”system:metrics-server” –requestheader-client-ca-file=/etc/kubernetes/cert/ca.pem –requestheader-extra-headers-prefix=”X-Remote-Extra-” –requestheader-group-headers=X-Remote-Group –requestheader-username-headers=X-Remote-User –cluster-signing-cert-file=/etc/kubernetes/cert/ca.pem –cluster-signing-key-file=/etc/kubernetes/cert/ca-key.pem –experimental-cluster-signing-duration=87600h –horizontal-pod-autoscaler-sync-period=10s –concurrent-deployment-syncs=10 –concurrent-gc-syncs=30 –node-cidr-mask-size=24 –service-cluster-ip-range=${SERVICE_CIDR} –cluster-cidr=${CLUSTER_CIDR} –pod-eviction-timeout=6m –terminated-pod-gc-threshold=10000 –root-ca-file=/etc/kubernetes/cert/ca.pem –service-account-private-key-file=/etc/kubernetes/cert/ca-key.pem –kubeconfig=/etc/kubernetes/kube-controller-manager.kubeconfig –logtostderr=true –v=2 Restart=on-failure RestartSec=5 [Install] WantedBy=multi-user.target EOF 5）启动并验证 [root@k8s-master01 work]# for master_ip in ${MASTER_IPS[@]} do echo “>>> ${master_ip}” scp kube-controller-manager.service.template root@${master_ip}:/etc/systemd/system/kube-controller-manager.service ssh root@${master_ip} “mkdir -p ${K8S_DIR}/kube-controller-manager” ssh root@${master_ip} “systemctl daemon-reload && systemctl enable kube-controller-manager –now” done查看输出的 Metrics
[root@k8s-master01 work]# curl -s –cacert /opt/k8s/work/ca.pem –cert /opt/k8s/work/admin.pem –key /opt/k8s/work/admin-key.pem https://127.0.0.1:10257/metrics | head查看权限
[root@k8s-master01 work]# kubectl describe clusterrole system:kube-controller-manager [root@k8s-master01 work]# kubectl get clusterrole | grep controller [root@k8s-master01 work]# kubectl describe clusterrole system:controller:deployment-controller查看当前的 Leader
[root@k8s-master01 work]# kubectl get endpoints kube-controller-manager –namespace=kube-system -o yaml3.安装 Kube-Scheduler 组件
1）创建 Kube-Scheduler 证书和密钥 [root@k8s-master01 work]# cat > kube-scheduler-csr.json << EOF { “CN”: “system:kube-scheduler”, “hosts”: [ “127.0.0.1”, “192.168.1.1”, “192.168.1.2” ], “key”: { “algo”: “rsa”, “size”: 2048 }, “names”: [ { “C”: “CN”, “ST”: “Shanghai”, “L”: “Shanghai”, “O”: “system:kube-scheduler”, “OU”: “System” } ] } EOF 2）生成证书和密钥 [root@k8s-master01 work]# cfssl gencert -ca=/opt/k8s/work/ca.pem -ca-key=/opt/k8s/work/ca-key.pem -config=/opt/k8s/work/ca-config.json -profile=kubernetes kube-scheduler-csr.json | cfssljson -bare kube-scheduler [root@k8s-master01 work]# for master_ip in ${MASTER_IPS[@]} do echo “>>> ${master_ip}” scp kube-scheduler*.pem root@${master_ip}:/etc/kubernetes/cert/ done 3）创建 Kubeconfig 文件 [root@k8s-master01 work]# kubectl config set-cluster kubernetes –certificate-authority=/opt/k8s/work/ca.pem –embed-certs=true –server=${KUBE_APISERVER} –kubeconfig=kube-scheduler.kubeconfig [root@k8s-master01 work]# kubectl config set-credentials system:kube-scheduler –client-certificate=kube-scheduler.pem –client-key=kube-scheduler-key.pem –embed-certs=true –kubeconfig=kube-scheduler.kubeconfig [root@k8s-master01 work]# kubectl config set-context system:kube-scheduler –cluster=kubernetes –user=system:kube-scheduler –kubeconfig=kube-scheduler.kubeconfig [root@k8s-master01 work]# kubectl config use-context system:kube-scheduler –kubeconfig=kube-scheduler.kubeconfig [root@k8s-master01 work]# for master_ip in ${MASTER_IPS[@]} do echo “>>> ${master_ip}” scp kube-scheduler.kubeconfig root@${master_ip}:/etc/kubernetes/ done 4）创建 Kube-Scheduler 配置文件 [root@k8s-master01 work]# cat > kube-scheduler.yaml.template << EOF apiVersion: kubescheduler.config.k8s.io/v1alpha1 kind: KubeSchedulerConfiguration bindTimeoutSeconds: 600 clientConnection: burst: 200 kubeconfig: “/etc/kubernetes/kube-scheduler.kubeconfig” qps: 100 enableContentionProfiling: false enableProfiling: true hardPodAffinitySymmetricWeight: 1 healthzBindAddress: 127.0.0.1:10251 leaderElection: leaderElect: true metricsBindAddress: 127.0.0.1:10251 EOF [root@k8s-master01 work]# for master_ip in ${MASTER_IPS[@]} do echo “>>> ${master_ip}” scp kube-scheduler.yaml.template root@${master_ip}:/etc/kubernetes/kube-scheduler.yaml done 5）创建启动脚本 [root@k8s-master01 work]# cat > kube-scheduler.service.template << EOF [Unit] Description=Kubernetes Scheduler Documentation=https://github.com/GoogleCloudPlatform/kubernetes [Service] WorkingDirectory=${K8S_DIR}/kube-scheduler ExecStart=/opt/k8s/bin/kube-scheduler –port=0 –secure-port=10259 –bind-address=127.0.0.1 –config=/etc/kubernetes/kube-scheduler.yaml –tls-cert-file=/etc/kubernetes/cert/kube-scheduler.pem –tls-private-key-file=/etc/kubernetes/cert/kube-scheduler-key.pem –authentication-kubeconfig=/etc/kubernetes/kube-scheduler.kubeconfig –client-ca-file=/etc/kubernetes/cert/ca.pem –requestheader-allowed-names=”system:metrics-server” –requestheader-client-ca-file=/etc/kubernetes/cert/ca.pem –requestheader-extra-headers-prefix=”X-Remote-Extra-” –requestheader-group-headers=X-Remote-Group –requestheader-username-headers=X-Remote-User –authorization-kubeconfig=/etc/kubernetes/kube-scheduler.kubeconfig –logtostderr=true –v=2 Restart=always RestartSec=5 StartLimitInterval=0 [Install] WantedBy=multi-user.target EOF 6）启动并验证 [root@k8s-master01 work]# for master_ip in ${MASTER_IPS[@]} do echo “>>> ${master_ip}” scp kube-scheduler.service.template root@${master_ip}:/etc/systemd/system/kube-scheduler.service ssh root@${master_ip} “mkdir -p ${K8S_DIR}/kube-scheduler” ssh root@${master_ip} “systemctl daemon-reload && systemctl enable kube-scheduler –now” done [root@k8s-master01 work]# netstat -nlpt | grep kube-schedule10251：接收 http 请求，非安全端口，不需要认证授权；
10259：接收 https 请求，安全端口，需要认认证授权（两个接口都对外提供 /metrics 和 /healthz 的访问）
查看输出的 Metrics
[root@k8s-master01 work]# curl -s –cacert /opt/k8s/work/ca.pem –cert /opt/k8s/work/admin.pem –key /opt/k8s/work/admin-key.pem https://127.0.0.1:10257/metrics | head查看权限
[root@k8s-master01 work]# kubectl describe clusterrole system:kube-controller-manager [root@k8s-master01 work]# kubectl get clusterrole | grep controller [root@k8s-master01 work]# kubectl describe clusterrole system:controller:deployment-controller查看当前的 Leader
[root@k8s-master01 work]# kubectl get endpoints kube-controller-manager –namespace=kube-system -o yaml4.安装 Kubelet 组件
[root@k8s-master01 work]# for all_ip in ${ALL_IPS[@]} do echo “>>> ${all_ip}” scp kubernetes/server/bin/kubelet root@${all_ip}:/opt/k8s/bin/ ssh root@${all_ip} “chmod +x /opt/k8s/bin/*” done [root@k8s-master01 work]# for all_name in ${ALL_NAMES[@]} do echo “>>> ${all_name}” export BOOTSTRAP_TOKEN=$(kubeadm token create –description kubelet-bootstrap-token –groups system:bootstrappers:${all_name} –kubeconfig ~/.kube/config) kubectl config set-cluster kubernetes –certificate-authority=/etc/kubernetes/cert/ca.pem –embed-certs=true –server=${KUBE_APISERVER} –kubeconfig=kubelet-bootstrap-${all_name}.kubeconfig kubectl config set-credentials kubelet-bootstrap –token=${BOOTSTRAP_TOKEN} –kubeconfig=kubelet-bootstrap-${all_name}.kubeconfig kubectl config set-context default –cluster=kubernetes –user=kubelet-bootstrap –kubeconfig=kubelet-bootstrap-${all_name}.kubeconfig kubectl config use-context default –kubeconfig=kubelet-bootstrap-${all_name}.kubeconfig done [root@k8s-master01 work]# kubeadm token list –kubeconfig ~/.kube/config # 查看 Kubeadm 为各节点创建的 Token [root@k8s-master01 work]# kubectl get secrets -n kube-system | grep bootstrap-token # 查看各 Token 关联的 Secret
创建 Kubelet 参数配置文件
[root@k8s-master01 work]# cat > kubelet-config.yaml.template << EOF kind: KubeletConfiguration apiVersion: kubelet.config.k8s.io/v1beta1 address: “##ALL_IP##” staticPodPath: “” syncFrequency: 1m fileCheckFrequency: 20s httpCheckFrequency: 20s staticPodURL: “” port: 10250 readOnlyPort: 0 rotateCertificates: true serverTLSBootstrap: true authentication: anonymous: enabled: false webhook: enabled: true x509: clientCAFile: “/etc/kubernetes/cert/ca.pem” authorization: mode: Webhook registryPullQPS: 0 registryBurst: 20 eventRecordQPS: 0 eventBurst: 20 enableDebuggingHandlers: true enableContentionProfiling: true healthzPort: 10248 healthzBindAddress: “##ALL_IP##” clusterDomain: “${CLUSTER_DNS_DOMAIN}” clusterDNS: – “${CLUSTER_DNS_SVC_IP}” nodeStatusUpdateFrequency: 10s nodeStatusReportFrequency: 1m imageMinimumGCAge: 2m imageGCHighThresholdPercent: 85 imageGCLowThresholdPercent: 80 volumeStatsAggPeriod: 1m kubeletCgroups: “” systemCgroups: “” cgroupRoot: “” cgroupsPerQOS: true cgroupDriver: cgroupfs runtimeRequestTimeout: 10m hairpinMode: promiscuous-bridge maxPods: 220 podCIDR: “${CLUSTER_CIDR}” podPidsLimit: -1 resolvConf: /etc/resolv.conf maxOpenFiles: 1000000 kubeAPIQPS: 1000 kubeAPIBurst: 2000 serializeImagePulls: false evictionHard: memory.available: “100Mi” nodefs.available: “10%” nodefs.inodesFree: “5%” imagefs.available: “15%” evictionSoft: {} enableControllerAttachDetach: true failSwapOn: true containerLogMaxSize: 20Mi containerLogMaxFiles: 10 systemReserved: {} kubeReserved: {} systemReservedCgroup: “” kubeReservedCgroup: “” enforceNodeAllocatable: [“pods”] EOF [root@k8s-master01 work]# for all_ip in ${ALL_IPS[@]} do echo “>>> ${all_ip}” sed -e “s/##ALL_IP##/${all_ip}/” kubelet-config.yaml.template > kubelet-config-${all_ip}.yaml.template scp kubelet-config-${all_ip}.yaml.template root@${all_ip}:/etc/kubernetes/kubelet-config.yaml done 1）创建 Kubelet 启动脚本 [root@k8s-master01 work]# cat > kubelet.service.template << EOF [Unit] Description=Kubernetes Kubelet Documentation=https://github.com/GoogleCloudPlatform/kubernetes After=docker.service Requires=docker.service [Service] WorkingDirectory=${K8S_DIR}/kubelet ExecStart=/opt/k8s/bin/kubelet –bootstrap-kubeconfig=/etc/kubernetes/kubelet-bootstrap.kubeconfig –cert-dir=/etc/kubernetes/cert –cgroup-driver=cgroupfs –cni-conf-dir=/etc/cni/net.d –container-runtime=docker –container-runtime-endpoint=unix:///var/run/dockershim.sock –root-dir=${K8S_DIR}/kubelet –kubeconfig=/etc/kubernetes/kubelet.kubeconfig –config=/etc/kubernetes/kubelet-config.yaml –hostname-override=##ALL_NAME## –pod-infra-container-image=registry.aliyuncs.com/google_containers/pause-amd64:3.2 –image-pull-progress-deadline=15m –volume-plugin-dir=${K8S_DIR}/kubelet/kubelet-plugins/volume/exec/ –logtostderr=true –v=2 Restart=always RestartSec=5 StartLimitInterval=0 [Install] WantedBy=multi-user.target EOF 2）启动并验证授权
[root@k8s-master01 ~]# kubectl create clusterrolebinding kubelet-bootstrap –clusterrole=system:node-bootstrapper –group=system:bootstrappers启动 Kubelet
[root@k8s-master01 work]# for all_name in ${ALL_NAMES[@]} do echo “>>> ${all_name}” ssh root@${all_name} “mkdir -p ${K8S_DIR}/kubelet/kubelet-plugins/volume/exec/” ssh root@${all_name} “systemctl daemon-reload && systemctl enable kubelet –now” done查看 Kubelet 服务
[root@k8s-master01 work]# for all_name in ${ALL_NAMES[@]} do echo “>>> ${all_name}” ssh root@${all_name} “systemctl status kubelet | grep active” done [root@k8s-master01 work]# kubectl get csr # 因为我们还没做认证. 所以显示 Pengding 状态
自动 Approve CSR 请求（创建三个 ClusterRoleBinding，分别用于自动 approve client renew client renew server 证书）
[root@k8s-master01 work]# cat > csr-crb.yaml << EOF # Approve all CSRs for the group “system:bootstrappers” kind: ClusterRoleBinding apiVersion: rbac.authorization.k8s.io/v1 metadata: name: auto-approve-csrs-for-group subjects: – kind: Group name: system:bootstrappers apiGroup: rbac.authorization.k8s.io roleRef: kind: ClusterRole name: system:certificates.k8s.io:certificatesigningrequests:nodeclient apiGroup: rbac.authorization.k8s.io — # To let a node of the group “system:nodes” renew its own credentials kind: ClusterRoleBinding apiVersion: rbac.authorization.k8s.io/v1 metadata: name: node-client-cert-renewal subjects: – kind: Group name: system:nodes apiGroup: rbac.authorization.k8s.io roleRef: kind: ClusterRole name: system:certificates.k8s.io:certificatesigningrequests:selfnodeclient apiGroup: rbac.authorization.k8s.io — # A ClusterRole which instructs the CSR approver to approve a node requesting a # serving cert matching its client cert. kind: ClusterRole apiVersion: rbac.authorization.k8s.io/v1 metadata: name: approve-node-server-renewal-csr rules: – apiGroups: [“certificates.k8s.io”] resources: [“certificatesigningrequests/selfnodeserver”] verbs: [“create”] — # To let a node of the group “system:nodes” renew its own server credentials kind: ClusterRoleBinding apiVersion: rbac.authorization.k8s.io/v1 metadata: name: node-server-cert-renewal subjects: – kind: Group name: system:nodes apiGroup: rbac.authorization.k8s.io roleRef: kind: ClusterRole name: approve-node-server-renewal-csr apiGroup: rbac.authorization.k8s.io EOF [root@k8s-master01 work]# kubectl apply -f csr-crb.yaml验证（等待一段时间 1 ~ 5 分钟)，三个节点的 CSR 都自动 approved）
[root@k8s-master01 work]# kubectl get csr | grep boot # 等待一段时间 (1-10 分钟)，三个节点的 CSR 都自动 approved [root@k8s-master01 work]# kubectl get nodes # 所有节点均 Ready
4）手动 Approve Server Cert Csr
基于安全性考虑，CSR approving controllers 不会自动 approve kubelet server 证书签名请求，需要手动 approve
[root@k8s-master01 ~]# kubectl get csr | grep node
Kubelet API 认证和授权
// 默认权限不足 [root@k8s-master01 ~]# curl -s –cacert /etc/kubernetes/cert/ca.pem –cert /etc/kubernetes/cert/kube-controller-manager.pem –key /etc/kubernetes/cert/kube-controller-manager-key.pem https://192.168.1.1:10250/metrics // 使用最高权限的 admin [root@k8s-master01 ~]# curl -s –cacert /etc/kubernetes/cert/ca.pem –cert /opt/k8s/work/admin.pem –key /opt/k8s/work/admin-key.pem https://192.168.1.1:10250/metrics | head
证书认证和授权
// 默认权限不足 [root@k8s-master01 ~]# curl -s –cacert /etc/kubernetes/cert/ca.pem –cert /etc/kubernetes/cert/kube-controller-manager.pem –key /etc/kubernetes/cert/kube-controller-manager-key.pem https://192.168.1.1:10250/metrics // 使用最高权限的 admin [root@k8s-master01 ~]# curl -s –cacert /etc/kubernetes/cert/ca.pem –cert /opt/k8s/work/admin.pem –key /opt/k8s/work/admin-key.pem https://192.168.1.1:10250/metrics | head创建 Bear Token 认证和授权
[root@k8s-master01 ~]# kubectl create serviceaccount kubelet-api-test [root@k8s-master01 ~]# kubectl create clusterrolebinding kubelet-api-test –clusterrole=system:kubelet-api-admin –serviceaccount=default:kubelet-api-test [root@k8s-master01 ~]# SECRET=$(kubectl get secrets | grep kubelet-api-test | awk {print $1}) [root@k8s-master01 ~]# TOKEN=$(kubectl describe secret ${SECRET} | grep -E ^token | awk {print $2}) [root@k8s-master01 ~]# echo ${TOKEN} [root@k8s-master01 ~]# curl -s –cacert /etc/kubernetes/cert/ca.pem -H “Authorization: Bearer ${TOKEN}” https://192.168.1.1:10250/metrics | head5.安装 Kube-Proxy 组件
Kube-Proxy 运行在所有主机上，用来监听 APIServer 中的 Service 和 Endpoint 的变化情况，并创建路由规则来提供服务 IP 和负载均衡功能。
[root@k8s-master01 work]# for all_ip in ${ALL_IPS[@]} do echo “>>> ${all_ip}” scp kubernetes/server/bin/kube-proxy root@${all_ip}:/opt/k8s/bin/ ssh root@${all_ip} “chmod +x /opt/k8s/bin/*” done1）创建 Kube-Proxy 证书和密钥
创建 Kube-Proxy 的 CA 证书请求文件
[root@k8s-master01 work]# cat > kube-proxy-csr.json << EOF { “CN”: “system:kube-proxy”, “key”: { “algo”: “rsa”, “size”: 2048 }, “names”: [ { “C”: “CN”, “ST”: “Shanghai”, “L”: “Shanghai”, “O”: “k8s”, “OU”: “System” } ] } EOF 2）生成证书和密钥 [root@k8s-master01 work]# cfssl gencert -ca=/opt/k8s/work/ca.pem -ca-key=/opt/k8s/work/ca-key.pem -config=/opt/k8s/work/ca-config.json -profile=kubernetes kube-proxy-csr.json | cfssljson -bare kube-proxy 3）创建 Kubeconfig 文件 [root@k8s-master01 work]# kubectl config set-cluster kubernetes –certificate-authority=/opt/k8s/work/ca.pem –embed-certs=true –server=${KUBE_APISERVER} –kubeconfig=kube-proxy.kubeconfig [root@k8s-master01 work]# kubectl config set-credentials kube-proxy –client-certificate=kube-proxy.pem –client-key=kube-proxy-key.pem –embed-certs=true –kubeconfig=kube-proxy.kubeconfig [root@k8s-master01 work]# kubectl config set-context default –cluster=kubernetes –user=kube-proxy –kubeconfig=kube-proxy.kubeconfig [root@k8s-master01 work]# kubectl config use-context default –kubeconfig=kube-proxy.kubeconfig [root@k8s-master01 work]# for all_ip in ${ALL_IPS[@]} do echo “>>> ${all_ip}” scp kube-proxy.kubeconfig root@${all_ip}:/etc/kubernetes/ done 4）创建 Kube-Proxy 配置文件 [root@k8s-master01 work]# cat > kube-proxy-config.yaml.template << EOF kind: KubeProxyConfiguration apiVersion: kubeproxy.config.k8s.io/v1alpha1 clientConnection: burst: 200 kubeconfig: “/etc/kubernetes/kube-proxy.kubeconfig” qps: 100 bindAddress: ##ALL_IP## healthzBindAddress: ##ALL_IP##:10256 metricsBindAddress: ##ALL_IP##:10249 enableProfiling: true clusterCIDR: ${CLUSTER_CIDR} hostnameOverride: ##ALL_NAME## mode: “ipvs” portRange: “” kubeProxyIPTablesConfiguration: masqueradeAll: false kubeProxyIPVSConfiguration: scheduler: rr excludeCIDRs: [] EOF [root@k8s-master01 work]# for (( i=0; i < 3; i++ )) do echo “>>> ${ALL_NAMES[i]}” sed -e “s/##ALL_NAME##/${ALL_NAMES[i]}/” -e “s/##ALL_IP##/${ALL_IPS[i]}/” kube-proxy-config.yaml.template > kube-proxy-config-${ALL_NAMES[i]}.yaml.template scp kube-proxy-config-${ALL_NAMES[i]}.yaml.template root@${ALL_NAMES[i]}:/etc/kubernetes/kube-proxy-config.yaml done4）创建启动脚本
[root@k8s-master01 work]# cat > kube-proxy.service << EOF [Unit] Description=Kubernetes Kube-Proxy Server Documentation=https://github.com/GoogleCloudPlatform/kubernetes After=network.target [Service] WorkingDirectory=${K8S_DIR}/kube-proxy ExecStart=/opt/k8s/bin/kube-proxy –config=/etc/kubernetes/kube-proxy-config.yaml –logtostderr=true –v=2 Restart=on-failure RestartSec=5 LimitNOFILE=65536 [Install] WantedBy=multi-user.target EOF [root@k8s-master01 work]# for all_name in ${ALL_NAMES[@]} do echo “>>> ${all_name}” scp kube-proxy.service root@${all_name}:/etc/systemd/system/ done 5）启动并验证 [root@k8s-master01 work]# for all_ip in ${ALL_IPS[@]} do echo “>>> ${all_ip}” ssh root@${all_ip} “mkdir -p ${K8S_DIR}/kube-proxy” ssh root@${all_ip} “modprobe ip_vs_rr” ssh root@${all_ip} “systemctl daemon-reload && systemctl enable kube-proxy –now” done查看 ipvs 路由规则
[root@k8s-master01 work]# ipvsadm -ln
问题： 当我们在启动 kube-proxy 组件后，通过 systemctl 查看该组件状态时，出现如下错误
Not using `–random-fully` in the MASQUERADE rule for iptables because the local version of iptables does not support it上面报错是因为我们的 iptables 版本不支持 –random-fully 配置（1.6.2 版本上支持），所以我们需要对 iptables 进行升级操作。
[root@master01 work]# wget https://www.netfilter.org/projects/iptables/files/iptables-1.6.2.tar.bz2 –no-check-certificate [root@master01 work]# for all_name in ${ALL_NAMES[@]} do echo “>>> ${all_name}” scp iptables-1.6.2.tar.bz2 root@${all_name}:/root/ ssh root@${all_name} “yum -y install gcc make libnftnl-devel libmnl-devel autoconf automake libtool bison flex libnetfilter_conntrack-devel libnetfilter_queue-devel libpcap-devel bzip2” ssh root@${all_name} “export LC_ALL=C && tar -xf iptables-1.6.2.tar.bz2 && cd iptables-1.6.2 && ./autogen.sh && ./configure && make && make install” ssh root@${all_name} “systemctl daemon-reload && systemctl restart kubelet && systemctl restart kube-proxy” done6.安装 CoreDNS 插件
1）修改 Coredns 配置 [root@k8s-master01 ~]# cd /opt/k8s/work/kubernetes/cluster/addons/dns/coredns [root@k8s-master01 coredns]# cp coredns.yaml.base coredns.yaml [root@k8s-master01 coredns]# sed -i -e “s/__PILLAR__DNS__DOMAIN__/${CLUSTER_DNS_DOMAIN}/” -e “s/__PILLAR__DNS__SERVER__/${CLUSTER_DNS_SVC_IP}/” -e “s/__PILLAR__DNS__MEMORY__LIMIT__/200Mi/” coredns.yaml 2）创建 Coredns 并启动配置调度策略
[root@k8s-master01 coredns]# vim coredns.yaml …… apiVersion: apps/v1 kind: Deployment …… spec: replicas: 2 # 配置成两个副本 …… tolerations: – key: “node-role.kubernetes.io/master” operator: “Equal” value: “” effect: NoSchedule nodeSelector: node-role.kubernetes.io/master: “true” …… [root@k8s-master01 coredns]# kubectl create -f coredns.yaml
因为上面镜像使用的是 K8s 官方的镜像（国外），所以可能会出现：
Normal BackOff 72s (x6 over 3m47s) kubelet, k8s-master01 Back-off pulling image “k8s.gcr.io/coredns:1.6.5” Warning Failed 57s (x7 over 3m47s) kubelet, k8s-master01 Error: ImagePullBackOff出现如上问题后，我们可以通过拉取其它仓库中的镜像，拉取完后重新打个标签即可。
如：docker pull k8s.gcr.io/coredns:1.6.5 我们可以： docker pull registry.aliyuncs.com/google_containers/coredns:1.6.5 docker tag registry.aliyuncs.com/google_containers/coredns:1.6.5 k8s.gcr.io/coredns:1.6.5 3）验证 [root@k8s-master01 coredns]# kubectl run -it –rm test-dns –image=busybox:1.28.4 sh If you dont see a command prompt, try pressing enter. / # / # nslookup kubernetes Server: 10.20.0.254 Address 1: 10.20.0.254 kube-dns.kube-system.svc.cluster.local Name: kubernetes Address 1: 10.20.0.1 kubernetes.default.svc.cluster.local7.安装 Dashboard 仪表盘
[root@k8s-master01 coredns]# cd /opt/k8s/work/ [root@k8s-master01 work]# mkdir metrics [root@k8s-master01 work]# cd metrics/ [root@k8s-master01 metrics]# wget https://github.com/kubernetes-sigs/metrics-server/releases/download/v0.3.6/components.yaml [root@k8s-master01 metrics]# vim components.yaml …… apiVersion: apps/v1 kind: Deployment metadata: name: metrics-server namespace: kube-system labels: k8s-app: metrics-server spec: replicas: 2 # 修改副本数 selector: matchLabels: k8s-app: metrics-server template: metadata: name: metrics-server labels: k8s-app: metrics-server spec: hostNetwork: true # 配置主机网络 serviceAccountName: metrics-server volumes: # mount in tmp so we can safely use from-scratch images and/or read-only containers – name: tmp-dir emptyDir: {} containers: – name: metrics-server image: registry.aliyuncs.com/google_containers/metrics-server-amd64:v0.3.6 # 修改镜像名 imagePullPolicy: IfNotPresent args: – –cert-dir=/tmp – –secure-port=4443 – –kubelet-insecure-tls # 新加的 – –kubelet-preferred-address-types=InternalIP,Hostname,InternalDNS,ExternalDNS,ExternalIP # 新加的 …… [root@k8s-master01 metrics]# kubectl create -f components.yaml验证：
手动创建 Secret
[root@master01 ~]# kubectl create namespace kubernetes-dashboard [root@master01 ~]# kubectl create secret generic kubernetes-dashboard-certs –from-file=/opt/k8s/work/dashboard/certs -n kubernetes-dashboard修改 Dashboard 配置（你们可以通过这个地址来看 Dashboard 的 yaml 文件：传送门）
[root@k8s-master01 work]# cd dashboard/ [root@k8s-master01 dashboard]# vim dashboard.yaml apiVersion: v1 kind: ServiceAccount metadata: labels: k8s-app: kubernetes-dashboard name: kubernetes-dashboard namespace: kubernetes-dashboard — kind: Service apiVersion: v1 metadata: labels: k8s-app: kubernetes-dashboard name: kubernetes-dashboard namespace: kubernetes-dashboard spec: type: NodePort ports: – port: 443 targetPort: 8443 nodePort: 30080 selector: k8s-app: kubernetes-dashboard — apiVersion: v1 kind: Secret metadata: labels: k8s-app: kubernetes-dashboard name: kubernetes-dashboard-csrf namespace: kubernetes-dashboard type: Opaque data: csrf: “” — apiVersion: v1 kind: Secret metadata: labels: k8s-app: kubernetes-dashboard name: kubernetes-dashboard-key-holder namespace: kubernetes-dashboard type: Opaque — kind: ConfigMap apiVersion: v1 metadata: labels: k8s-app: kubernetes-dashboard name: kubernetes-dashboard-settings namespace: kubernetes-dashboard — kind: Role apiVersion: rbac.authorization.k8s.io/v1 metadata: labels: k8s-app: kubernetes-dashboard name: kubernetes-dashboard namespace: kubernetes-dashboard rules: – apiGroups: [“”] resources: [“secrets”] resourceNames: [“kubernetes-dashboard-key-holder”, “kubernetes-dashboard-certs”, “kubernetes-dashboard-csrf”] verbs: [“get”, “update”, “delete”] – apiGroups: [“”] resources: [“configmaps”] resourceNames: [“kubernetes-dashboard-settings”] verbs: [“get”, “update”] – apiGroups: [“”] resources: [“services”] resourceNames: [“heapster”, “dashboard-metrics-scraper”] verbs: [“proxy”] – apiGroups: [“”] resources: [“services/proxy”] resourceNames: [“heapster”, “http:heapster:”, “https:heapster:”, “dashboard-metrics-scraper”, “http:dashboard-metrics-scraper”] verbs: [“get”] — kind: ClusterRole apiVersion: rbac.authorization.k8s.io/v1 metadata: labels: k8s-app: kubernetes-dashboard name: kubernetes-dashboard rules: # Allow Metrics Scraper to get metrics from the Metrics server – apiGroups: [“metrics.k8s.io”] resources: [“pods”, “nodes”] verbs: [“get”, “list”, “watch”] — apiVersion: rbac.authorization.k8s.io/v1 kind: RoleBinding metadata: labels: k8s-app: kubernetes-dashboard name: kubernetes-dashboard namespace: kubernetes-dashboard roleRef: apiGroup: rbac.authorization.k8s.io kind: Role name: kubernetes-dashboard subjects: – kind: ServiceAccount name: kubernetes-dashboard namespace: kubernetes-dashboard — apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRoleBinding metadata: name: kubernetes-dashboard roleRef: apiGroup: rbac.authorization.k8s.io kind: ClusterRole name: kubernetes-dashboard subjects: – kind: ServiceAccount name: kubernetes-dashboard namespace: kubernetes-dashboard — kind: Deployment apiVersion: apps/v1 metadata: labels: k8s-app: kubernetes-dashboard name: kubernetes-dashboard namespace: kubernetes-dashboard spec: replicas: 1 revisionHistoryLimit: 10 selector: matchLabels: k8s-app: kubernetes-dashboard template: metadata: labels: k8s-app: kubernetes-dashboard spec: containers: – name: kubernetes-dashboard image: kubernetesui/dashboard:v2.0.0-beta8 imagePullPolicy: IfNotPresent ports: – containerPort: 8443 protocol: TCP args: – –auto-generate-certificates – –namespace=kubernetes-dashboard – –tls-key-file=tls.key – –tls-cert-file=tls.crt – –token-ttl=3600 volumeMounts: – name: kubernetes-dashboard-certs mountPath: /certs – mountPath: /tmp name: tmp-volume livenessProbe: httpGet: scheme: HTTPS path: / port: 8443 initialDelaySeconds: 30 timeoutSeconds: 30 securityContext: allowPrivilegeEscalation: false readOnlyRootFilesystem: true runAsUser: 1001 runAsGroup: 2001 volumes: – name: kubernetes-dashboard-certs secret: secretName: kubernetes-dashboard-certs – name: tmp-volume emptyDir: {} serviceAccountName: kubernetes-dashboard nodeSelector: “beta.kubernetes.io/os”: linux tolerations: – key: node-role.kubernetes.io/master effect: NoSchedule — kind: Service apiVersion: v1 metadata: labels: k8s-app: dashboard-metrics-scraper name: dashboard-metrics-scraper namespace: kubernetes-dashboard spec: ports: – port: 8000 targetPort: 8000 selector: k8s-app: dashboard-metrics-scraper — kind: Deployment apiVersion: apps/v1 metadata: labels: k8s-app: dashboard-metrics-scraper name: dashboard-metrics-scraper namespace: kubernetes-dashboard spec: replicas: 1 revisionHistoryLimit: 10 selector: matchLabels: k8s-app: dashboard-metrics-scraper template: metadata: labels: k8s-app: dashboard-metrics-scraper annotations: seccomp.security.alpha.kubernetes.io/pod: runtime/default spec: containers: – name: dashboard-metrics-scraper image: kubernetesui/metrics-scraper:v1.0.1 imagePullPolicy: IfNotPresent ports: – containerPort: 8000 protocol: TCP livenessProbe: httpGet: scheme: HTTP path: / port: 8000 initialDelaySeconds: 30 timeoutSeconds: 30 volumeMounts: – mountPath: /tmp name: tmp-volume securityContext: allowPrivilegeEscalation: false readOnlyRootFilesystem: true runAsUser: 1001 runAsGroup: 2001 serviceAccountName: kubernetes-dashboard nodeSelector: “beta.kubernetes.io/os”: linux tolerations: – key: node-role.kubernetes.io/master effect: NoSchedule volumes: – name: tmp-volume emptyDir: {} [root@k8s-master01 dashboard]# kubectl create -f dashboard.yaml创建管理员账户
[root@k8s-master01 dashboard]# kubectl create serviceaccount admin-user -n kubernetes-dashboard [root@k8s-master01 dashboard]# kubectl create clusterrolebinding admin-user –clusterrole=cluster-admin –serviceaccount=kubernetes-dashboard:admin-user 3）验证获取登录令牌
[root@k8s-master01 dashboard]# kubectl -n kubernetes-dashboard describe secret $(kubectl -n kubernetes-dashboard get secret | grep admin-user | awk {print $1})
访问：https://192.168.1.1:30080
到此，我们的 Kubernetes 便搭建完成了，如果你们在安装中出现问题，可以提出，大家共同讨论，希望大家以后多多支持！
免费资源网 – https://freexyz.cn/
© 版权声明
THE END
暂无评论内容