Kubernetes Ansible 部署生产级别高可用的集群

用户11081884

发布于 2026-07-20 19:00:36

290

云原生环境中Kubernetes 已成为容器编排的事实标准。对于生产环境而言，高可用性是不可或缺的关键特性。本文将介绍如何使用 Ansible Playbook 部署一个生产级别的高可用 Kubernetes 集群，涵盖从系统初始化到集群运维的全流程。

Kubernetes 集群核心组件：

多节点 etcd 集群（至少3节点）
多 Master 节点配合负载均衡
可扩展的 Worker 节点
双栈网络支持
证书自动管理

环境准备

1. 主机规划

创建 inventory 文件定义集群架构：

# etcd 集群节点
[etcd]
192.168.1.10 hostname=etcd-01
192.168.1.11 hostname=etcd-02
192.168.1.12 hostname=etcd-03


# 负载均衡节点
[loadbalancer]
192.168.1.20 hostname=lb-01
192.168.1.21 hostname=lb-02


# Master 节点
[master]
192.168.1.30 hostname=master-01
192.168.1.31 hostname=master-02
192.168.1.32 hostname=master-03


# Worker 节点
[worker]
192.168.1.40 hostname=worker-01
192.168.1.41 hostname=worker-02
192.168.1.42 hostname=worker-03


# 集群所有节点
[k8s-cluster:children]
etcd
master
worker

2. 变量配置

创建 group_vars/all.yml 配置文件：

# 集群基础配置
cluster_name: production-k8s
kubernetes_version: "1.28.4"
service_cidr: "10.96.0.0/16"
pod_cidr: "10.244.0.0/16"
network_plugin: calico


# 证书配置
cert_validity_days: 3650
cert_dir: "/etc/kubernetes/pki"


# 系统配置
system_reserved: true
system_swap: false
selinux_status: enforcing


# 容器运行时
container_runtime: containerd
containerd_version: "1.7.11"

部署流程

1. 系统初始化

创建系统初始化 Playbook：

# playbooks/system-init.yml
- name: 初始化 Kubernetes 集群节点
  hosts: k8s-cluster
  become: yes
  tasks:
    - name: 禁用 SELinux 临时
      selinux:
        state: permissive
      when: selinux_status == "enforcing"


    - name: 关闭交换分区
      shell: |
        swapoff -a
        sed -i '/ swap / s/^\(.*\)$/#\1/g' /etc/fstab


    - name: 加载内核模块
      modprobe:
        name: "{{ item }}"
        state: present
      loop:
        - br_netfilter
        - overlay
        - ip_vs
        - ip_vs_rr
        - ip_vs_wrr
        - ip_vs_sh


    - name: 配置系统参数
      sysctl:
        name: "{{ item.name }}"
        value: "{{ item.value }}"
        state: present
        reload: yes
      loop:
        - { name: net.bridge.bridge-nf-call-iptables, value: 1 }
        - { name: net.ipv4.ip_forward, value: 1 }
        - { name: net.bridge.bridge-nf-call-ip6tables, value: 1 }

2. 证书管理

创建证书生成 Playbook：

# playbooks/certificates.yml
- name: 生成集群 TLS 证书
  hosts: localhost
  become: yes
  vars:
    ca_cert_path: "{{ cert_dir }}/ca.crt"
    ca_key_path: "{{ cert_dir }}/ca.key"
  tasks:
    - name: 创建证书目录
      file:
        path: "{{ cert_dir }}"
        state: directory
        mode: 0755


    - name: 生成 CA 证书
      openssl_certificate:
        path: "{{ ca_cert_path }}"
        privatekey_path: "{{ ca_key_path }}"
        common_name: "Kubernetes CA"
        basic_constraints_critical: yes
        basic_constraints: "CA:TRUE"
        key_usage_critical: yes
        key_usage:
          - keyCertSign
          - digitalSignature
        extended_key_usage:
          - serverAuth
          - clientAuth


    - name: 生成 API Server 证书
      openssl_certificate:
        path: "{{ cert_dir }}/apiserver.crt"
        privatekey_path: "{{ cert_dir }}/apiserver.key"
        csr_path: "{{ cert_dir }}/apiserver.csr"
        common_name: "kube-apiserver"
        subject_alt_name: "DNS:kubernetes,DNS:kubernetes.default,DNS:kubernetes.default.svc,DNS:kubernetes.default.svc.cluster.local,DNS:{{ inventory_hostname }},IP:127.0.0.1,IP:{{ service_cidr.split('/')[0] }}"
        ca_cert_path: "{{ ca_cert_path }}"
        ca_privatekey_path: "{{ ca_key_path }}"

3. etcd 集群部署

# playbooks/etcd-cluster.yml
- name: 部署 etcd 集群
  hosts: etcd
  become: yes
  vars:
    etcd_version: "3.5.10"
    etcd_data_dir: "/var/lib/etcd"
  tasks:
    - name: 下载 etcd 二进制文件
      get_url:
        url: " https://github.com/etcd-io/etcd/releases/download/v {{ etcd_version }}/etcd-v{{ etcd_version }}-linux-amd64.tar.gz"
        dest: "/tmp/etcd.tar.gz"


    - name: 解压 etcd
      unarchive:
        src: "/tmp/etcd.tar.gz"
        dest: "/usr/local/bin"
        remote_src: yes
        owner: root
        group: root
        mode: 0755


    - name: 创建 etcd 数据目录
      file:
        path: "{{ etcd_data_dir }}"
        state: directory
        owner: etcd
        group: etcd
        mode: 0755


    - name: 配置 etcd 服务
      template:
        src: templates/etcd.service.j2
        dest: /etc/systemd/system/etcd.service
        mode: 0644
      notify: reload systemd


    - name: 启动 etcd 服务
      systemd:
        name: etcd
        state: started
        enabled: yes
        daemon_reload: yes

4. 负载均衡配置

# playbooks/loadbalancer.yml
- name: 配置 HAProxy 负载均衡
  hosts: loadbalancer
  become: yes
  tasks:
    - name: 安装 HAProxy
      package:
        name: haproxy
        state: present


    - name: 配置 HAProxy
      template:
        src: templates/haproxy.cfg.j2
        dest: /etc/haproxy/haproxy.cfg
        mode: 0644
      vars:
        master_nodes: "{{ groups['master'] }}"
        api_port: 6443


    - name: 启动 HAProxy
      systemd:
        name: haproxy
        state: started
        enabled: yes

5. Kubernetes 组件部署

创建 Master 节点部署 Playbook：

# playbooks/kube-master.yml
- name: 部署 Kubernetes Master 组件
  hosts: master
  become: yes
  tasks:
    - name: 安装 kubelet
      package:
        name: kubelet
        state: present


    - name: 安装 kubeadm kubectl
      package:
        name: "{{ item }}"
        state: present
      loop:
        - kubeadm
        - kubectl


    - name: 配置 kubelet
      template:
        src: templates/kubelet-config.yaml.j2
        dest: /var/lib/kubelet/config.yaml
        mode: 0644


    - name: 启动 kubelet
      systemd:
        name: kubelet
        state: started
        enabled: yes


    - name: 初始化第一个 Master 节点
      shell: |
        kubeadm init \
          --control-plane-endpoint "{{ lb_vip }}:6443" \
          --upload-certs \
          --pod-network-cidr {{ pod_cidr }} \
          --service-cidr {{ service_cidr }}
      when: inventory_hostname == groups['master']

集群运维

1. 节点扩容

# 添加新 Worker 节点
ansible-playbook -i inventory playbooks/scale-worker.yml \
  -e "new_worker=192.168.1.43" \
  -l 192.168.1.43

2. 证书轮换

# playbooks/cert-rotation.yml
- name: 轮换 Kubernetes 证书
  hosts: master
  become: yes
  tasks:
    - name: 备份现有证书
      command: |
        tar -czf /tmp/k8s-certs-backup-$(date +%Y%m%d).tar.gz {{ cert_dir }}
      
    - name: 生成新证书
      command: |
        kubeadm certs renew all
        
    - name: 重启控制平面组件
      systemd:
        name: "{{ item }}"
        state: restarted
      loop:
        - kube-apiserver
        - kube-controller-manager
        - kube-scheduler

3. 版本升级

# 升级 Kubernetes 版本
ansible-playbook -i inventory playbooks/upgrade-cluster.yml \
  -e "target_version=1.29.0" \
  -l master

验证集群状态

部署完成后，使用以下命令验证集群健康状态：

# 检查节点状态
kubectl get nodes -o wide
# 检查组件状态
kubectl get componentstatuses
# 检查 Pod 状态
kubectl get pods -n kube-system
# 验证高可用性
kubectl get endpoints kubernetes -o yaml

通过 Ansible 自动化部署高可用 Kubernetes 集群，不仅提高了部署效率，还确保了环境的一致性和可重复性，部署方式特别适合需要频繁部署、维护多个 Kubernetes 集群的企业环境，为云原生应用的稳定运行提供了坚实的基础设施保障。

“无他，惟手熟尔”！有需要的用起来！

如果你觉得这篇文章有用，欢迎点赞、转发、收藏、留言、推荐❤！

本文参与腾讯云自媒体同步曝光计划，分享自微信公众号。

原始发表：2025-11-10，如有侵权请联系 cloudcommunity@tencent.com 删除

集群