k8s-学习笔记2-容器部署nginx负载均衡

deployment文件部署

# cat nginx_deployment
apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx
spec:
  replicas: 2
  selector:
    matchLabels:
      app: nginx
  strategy:
    type: RollingUpdate
  template:
    metadata:
      labels:
        app: nginx
    spec:
      containers:
        - image: nginx
          imagePullPolicy: Always
          name: nginx
      imagePullSecrets:
        - name: default-secret

部署完成后，查看pod信息，可以看到两个pod的ip，和所在的node。

# kubectl get pods -l app=nginx -o wide
NAME                     READY   STATUS    RESTARTS   AGE   IP           NODE                          NOMINATED NODE   READINESS GATES
nginx-645ccd8dc9-fmrtb   1/1     Running   0          72m   10.244.1.9   192.168.18.5-hd1   <none>           <none>
nginx-645ccd8dc9-w75cd   1/1     Running   0          72m   10.244.1.8   192.168.18.5-hd1   <none>           <none>

因为是副本部署，所以当一个pod被删除，就会自动创建新的pod

可见，删除一个pod的下一刻，新的pod已经被创建，但还没准备好，也没有分配ip

# kubectl delete pod nginx-645ccd8dc9-fmrtb
pod "nginx-645ccd8dc9-fmrtb" deleted

# kubectl get pods -l app=nginx -o wide    
NAME                     READY   STATUS              RESTARTS   AGE     IP           NODE                          NOMINATED NODE   READINESS GATES
nginx-645ccd8dc9-pr8l7   0/1     ContainerCreating   0          5s      <none>       kibanaprod-192.168.18.5-hd1   <none>           <none>
nginx-645ccd8dc9-w75cd   1/1     Running             0          4h28m   10.244.1.8   kibanaprod-192.168.18.5-hd1   <none>           <none>

过几秒钟就已经分配好，这个新的pod ip和原来的是不一样的。

# kubectl get pods -l app=nginx -o wide
NAME                     READY   STATUS    RESTARTS   AGE     IP            NODE                          NOMINATED NODE   READINESS GATES
nginx-645ccd8dc9-pr8l7   1/1     Running   0          112s    10.244.1.13   kibanaprod-192.168.18.5-hd1   <none>           <none>
nginx-645ccd8dc9-w75cd   1/1     Running   0          4h30m   10.244.1.8    kibanaprod-192.168.18.5-hd1   <none>           <none>

curl pod ip可以直接访问

从k8s master上traceroute发现是直接通过flannel网络达到位于另一个node上的pod。

# curl 10.244.1.9

working. Further configuration is required.</p>

[root@masterpub-10.0.0.80-hd1 18:03 /data/k8s/nginx]
# traceroute 10.244.1.9
traceroute to 10.244.1.9 (10.244.1.9), 30 hops max, 60 byte packets
 1  10.244.1.0 (10.244.1.0)  1.321 ms *  1.225 ms
 2  10.244.1.9 (10.244.1.9)  1.290 ms  1.248 ms  1.212 ms

service expose

--port svc端口

--target-port 容器端口

--type 类型　　ClusterIP, NodePort, LoadBalancer, or ExternalName. Default is 'ClusterIP'

一个svc，将一个deployment对接起来

10.99.188.113这个地址是动态生成的，它只有在集群内部才能访问到。但master上无法访问，在node上使用curl就能访问。

# kubectl get deployment       
NAME    READY   UP-TO-DATE   AVAILABLE   AGE
nginx   2/2     2            2           4h42m

# kubectl expose deployment nginx --name nginx-svc --port=80 --target-port=80
service/nginx-svc exposed

# kubectl get svc
NAME         TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)   AGE
kubernetes   ClusterIP   10.96.0.1       <none>        443/TCP   29d
nginx-svc    ClusterIP   10.99.188.113   <none>        80/TCP    92s

查看svc的详细信息

可以看到它的名称，名称空间，标签选择器，类型，ip，endpoint

# kubectl describe svc nginx-svc
Name:              nginx-svc
Namespace:         default
Labels:            <none>
Annotations:       <none>
Selector:          app=nginx
Type:              ClusterIP
IP:                10.99.188.113
Port:              <unset>  80/TCP
TargetPort:        80/TCP
Endpoints:         10.244.1.15:80,10.244.1.16:80
Session Affinity:  None
Events:            <none>

动态修改svc的type，改成NodePort，这样外网就能访问到

# kubectl edit svc nginx-svc

# kubectl get svc           
NAME         TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)        AGE
kubernetes   ClusterIP   10.96.0.1       <none>        443/TCP        29d
nginx-svc    NodePort    10.99.188.113   <none>        80:31621/TCP   59m

这时外网访问k8s集群中任意一个node:port，都能连接到svc。

可以做一个外部的内网SLB，绑定一个外网，把所有node接入。

scale扩容

如果两个副本不够用，可以动态扩展

# kubectl scale --replicas=4 deployment nginx
deployment.apps/nginx scaled

# kubectl get pods -l app=nginx
NAME                     READY   STATUS    RESTARTS   AGE
nginx-645ccd8dc9-9xkwd   1/1     Running   0          16m
nginx-645ccd8dc9-nfthl   1/1     Running   0          16m
nginx-645ccd8dc9-q6nlt   1/1     Running   0          82s
nginx-645ccd8dc9-x4gbk   1/1     Running   0          82s

也可以缩容，指定副本数，又只剩下2个了

# kubectl scale --replicas=2 deployment nginx 
deployment.apps/nginx scaled
[root@masterpub-10.0.0.80-hd1 22:14 ~]

NAME                     READY   STATUS    RESTARTS   AGE
nginx-645ccd8dc9-9xkwd   1/1     Running   0          18m
nginx-645ccd8dc9-nfthl   1/1     Running   0          18m

更新/回滚deployment

查看目前的deployment，image是nginx

# kubectl describe deployment nginx

  Containers:
   nginx:
    Image:        nginx
    Port:         <none>
    Host Port:    <none>
    Environment:  <none>
    Mounts:       <none>
  Volumes:        <none>

查看目前deployment相关的pods，都在运行

# kubectl get pods -l app=nginx
NAME                     READY   STATUS    RESTARTS   AGE
nginx-645ccd8dc9-fj2kk   1/1     Running   0          3m25s
nginx-645ccd8dc9-jfspl   1/1     Running   0          3m25s
nginx-645ccd8dc9-l9686   1/1     Running   0          21m
nginx-645ccd8dc9-nb8zl   1/1     Running   0          21m

更新image

# kubectl set image deployment/nginx nginx=nginx:1.9.1
deployment.apps/nginx image updated

# kubectl describe deployment nginx

  Containers:
   nginx:
    Image:        nginx:1.9.1
    Port:         <none>
    Host Port:    <none>
    Environment:  <none>
    Mounts:       <none>
  Volumes:        <none>

第一次查看时，pods正在滚动替换，创建新的pod，销毁旧的。

第二次查看时，已全部替换

# kubectl get pods -l app=nginx
NAME                     READY   STATUS              RESTARTS   AGE
nginx-544b8d8974-78k8d   0/1     ContainerCreating   0          5s
nginx-544b8d8974-g28kz   1/1     Running             0          24s
nginx-544b8d8974-tlx4h   0/1     ContainerCreating   0          24s
nginx-645ccd8dc9-jfspl   1/1     Running             0          4m19s
nginx-645ccd8dc9-l9686   0/1     Terminating         0          22m
nginx-645ccd8dc9-nb8zl   1/1     Running             0          22m

# kubectl get pods -l app=nginx
NAME                     READY   STATUS    RESTARTS   AGE
nginx-544b8d8974-78k8d   1/1     Running   0          2m54s
nginx-544b8d8974-g28kz   1/1     Running   0          3m13s
nginx-544b8d8974-gnhqk   1/1     Running   0          2m33s
nginx-544b8d8974-tlx4h   1/1     Running   0          3m13s

回滚刚才的更新

# kubectl rollout undo deployment/nginx  
deployment.apps/nginx rolled back

# kubectl rollout status deployment/nginx
Waiting for deployment "nginx" rollout to finish: 2 out of 4 new replicas have been updated...
Waiting for deployment "nginx" rollout to finish: 2 out of 4 new replicas have been updated...
Waiting for deployment "nginx" rollout to finish: 2 out of 4 new replicas have been updated...
Waiting for deployment "nginx" rollout to finish: 3 out of 4 new replicas have been updated...
Waiting for deployment "nginx" rollout to finish: 3 out of 4 new replicas have been updated...
Waiting for deployment "nginx" rollout to finish: 3 out of 4 new replicas have been updated...
Waiting for deployment "nginx" rollout to finish: 3 out of 4 new replicas have been updated...
Waiting for deployment "nginx" rollout to finish: 1 old replicas are pending termination...
Waiting for deployment "nginx" rollout to finish: 1 old replicas are pending termination...
Waiting for deployment "nginx" rollout to finish: 1 old replicas are pending termination...
Waiting for deployment "nginx" rollout to finish: 3 of 4 updated replicas are available...
deployment "nginx" successfully rolled out

coredns

service提供一个ip，但外界访问需要一个域名，这个解析的工作就是coredns来完成

# kubectl get pods -o wide -n kube-system|grep dns
coredns-7ff77c879f-6wkt6                          1/1     Running   0          29d   10.244.0.2     masterpub-10.0.0.80-hd1       <none>           <none>
coredns-7ff77c879f-m2crn                          1/1     Running   0          29d   10.244.0.3     masterpub-10.0.0.80-hd1       <none>           <none>

# kubectl get svc -n kube-system
NAME       TYPE        CLUSTER-IP   EXTERNAL-IP   PORT(S)                  AGE
kube-dns   ClusterIP   10.96.0.10   <none>        53/UDP,53/TCP,9153/TCP   29d

部署一个交互式的busybox测试，查看它的dns解析，可见就是coredns的svc地址

# kubectl run client-test --image=busybox -it --restart=Never
If you don't see a command prompt, try pressing enter.
/ # cat /etc/resolv.conf 
nameserver 10.96.0.10
search default.svc.cluster.local svc.cluster.local cluster.local
options ndots:5

访问svc的域名，可见，已经成功解析

/ # wget nginx-svc
Connecting to nginx-svc (10.99.188.113:80)

尝试删除deployment里的两个pod，然后再测试在pod客户端里访问域名

前面10s左右无法通信，后面就有了，说明deployment里的pod已经重新创建，svc不管后面怎么变化，都通信正常

/ # wget -O - -q http://nginx-svc
wget: can't connect to remote host (10.99.188.113): Connection refused
/ # wget -O - -q http://nginx-svc
wget: can't connect to remote host (10.99.188.113): Connection refused
/ # wget -O - -q http://nginx-svc
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>

service文件部署

改成NodePort，就是映射到node的port上。这里没有写nodeport的值，就是随机，范围是30000多。

文件中使用标签选择器，就是将app:nginx的pod纳入service。

# cat nginx_deployment_svc
apiVersion: v1
kind: Service
metadata:
  name: nginxservice
spec:
  type: NodePort
  selector:
    app: nginx
  ports:
    - name: http
      protocol: TCP
      port: 80
      targetPort: 80
    - name: https
      protocol: TCP
      port: 443
      targetPort: 443

查看service的信息，看到type，集群ip，端口映射。

查看这个service的endpoint信息，可见将两个pod已经纳入

# kubectl get svc
NAME           TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)                      AGE
nginxservice   NodePort    10.96.65.135    <none>        80:30205/TCP,443:32418/TCP   64m

# kubectl get ep nginxservice
NAME           ENDPOINTS                                                 AGE
nginxservice   10.244.1.8:443,10.244.1.9:443,10.244.1.8:80 + 1 more...   66m

访问顺序

这时外网就从node ip访问nginx service

先到node ip，再到cluster ip，再到pod ip

从仓库中拉取镜像部署

创建一个namespace

#cat namespace.yaml

apiVersion: v1

kind: Namespace

metadata:

name: redis

kubectl apply -f namespace.yaml

kubectl get namespace

创建一个configmap

redis.conf

kubectl create configmap redis-conf --from-file=redis.conf -n redis

kubectl get configmap -n redis