k8s- nginx ingress 高可用部署，最新版，支持 k8s 1.22---1.19第2篇
nginx-ingress-controller 安装完毕，接下来开始对 nginx-ingress-controller 实现高可用。我们通过 keepalive+nginx 实现 nginx-ingress-controller 高可用！
注意：这里的keepalive+nginx 仅仅是对 nginx-ingress-controller 实现高可用！本教程里都是在Worker节点操作，即安装了nginx-ingress-controller 的节点操作。
一、安装keepalive+nginx
3台安装了nginx-ingress-controller的worker节点上分别安装。
[root@k8snode1 mytest]# yum install nginx keepalived -y
[root@k8snode2 mytest]# yum install nginx keepalived -y
[root@k8snode3 mytest]# yum install nginx keepalived -y
二、修改 nginx 配置文件。主备一样
vim /etc/nginx/nginx.conf
user nginx;
worker_processes auto;
error_log /var/log/nginx/error.log;
pid /run/nginx.pid;
include /usr/share/nginx/modules/*.conf;
events {
worker_connections 1024;
}
# 四层负载均衡，为两台 Master apiserver 组件提供负载均衡
stream {
log_format main '$remote_addr $upstream_addr - [$time_local] $status $upstream_bytes_sent';
access_log /var/log/nginx/k8s-access.log main;
upstream k8s-apiserver {
server 192.168.157.202:80; # Master1 APISERVER IP:PORT
server 192.168.157.203:80; # Master2 APISERVER IP:PORT
server 192.168.157.204:80; # Master2 APISERVER IP:PORT
}
server {
listen 10080;
proxy_pass k8s-apiserver;
}
}
http {
log_format main '$remote_addr - $remote_user [$time_local] "$request" '
'$status $body_bytes_sent "$http_referer" '
'"$http_user_agent" "$http_x_forwarded_for"';
access_log /var/log/nginx/access.log main;
sendfile on;
tcp_nopush on;
tcp_nodelay on;
keepalive_timeout 65;
types_hash_max_size 2048;
include /etc/nginx/mime.types;
default_type application/octet-stream;
server {
listen 8080 default_server;
server_name _;
location / {
}
}
}
请将 upstream k8s-apiserver里的ip设置为安装了 nginx-ingress controller对应的宿主机的IP地址。端口号为80.
三、keepalive 配置
双机热备，其中主节点和备节点的配置稍稍有所不同。
vim /etc/keepalived/keepalived.conf
global_defs {
notification_email {
acassen@firewall.loc
failover@firewall.loc
sysadmin@firewall.loc
}
notification_email_from Alexandre.Cassen@firewall.loc
smtp_server 127.0.0.1
smtp_connect_timeout 30
router_id NGINX_MASTER
}
vrrp_script check_nginx {
script "/etc/keepalived/check_nginx.sh"
}
vrrp_instance VI_1 {
state MASTER
interface ens33 # 修改为实际网卡名
virtual_router_id 51 # VRRP 路由 ID实例，每个实例是唯一的
priority 100 # 优先级，备服务器设置 90
advert_int 1 # 指定VRRP 心跳包通告间隔时间，默认1秒
authentication {
auth_type PASS
auth_pass 1111
}
# 虚拟IP
virtual_ipaddress {
192.168.157.199/24
}
track_script {
check_nginx
}
}
#vrrp_script：指定检查nginx工作状态脚本（根据nginx状态判断是否故障转移）
#virtual_ipaddress：虚拟IP（VIP）需要修改4个地方：
1、 interface +本机的网卡名
2、state MASTER ，若为主节点则为MASTER，备节点为BACKUP ,都是大写字母。
3、priority 100 优先级，主节点MASTER则设置100，备节点设置比100小即可，比如90 .
4、virtual_ipaddress 虚拟IP，设置为当前主机环境一个未被使用的IP。
从该配置文件可以看出，还需要创建一个check_nginx.sh 文件。
三、创建check_nginx.sh
vim /etc/keepalived/check_nginx.sh
#!/bin/bash
#1、判断Nginx是否存活
counter=`ps -C nginx --no-header | wc -l`
if [ $counter -eq 0 ]; then
#2、如果不存活则尝试启动Nginx
service nginx start
sleep 2
#3、等待2秒后再次获取一次Nginx状态
counter=`ps -C nginx --no-header | wc -l`
#4、再次进行判断，如Nginx还不存活则停止Keepalived，让地址进行漂移
if [ $counter -eq 0 ]; then
service keepalived stop
fi
fi四、3台机器上执行
1、需要安装一个插件
yum install nginx-mod-stream -y
2、开启
systemctl daemon-reload
systemctl enable nginx keepalived
systemctl start nginx
systemctl start keepalived
教程中涉及到的文件可以下载：
链接：https://pan.baidu.com/s/1oRvhN2_nfVT2ndE2VEN2QQ
提取码：muxx