[K8S] Ingress Controller

인그레스란?

HTTP나 HTTPS를 통해 클러스터 내부의 서비스를 외부로 노출
- Service에 외부 URL 제공
- 트래픽을 로드밸런싱
- SSL 인증서 처리 (HTTPS)
- Virtual Hosting 지정

👉 서비스들에 대한 단일 진입점을 만들어 내부의 서비스를 외부에서 접속 가능하도록 해준다.

서비스 Rule을 지정해서 Ingress Controller에 넣는다.
ex) Multi Path 
http://www.examle.com/                -->svc Main
http://www.examle.com/login        -->svc Login
http://www.examle.com/order        -->svc Order

외부 클라이언트의 접속 URL이 뭔지에 따라 연결되는 Service가 다르다.

https://kubernetes.io/docs/concepts/services-networking/ingress/

Ingress

# Ingress Controller 설치

Kubernetes as a project supports and maintains AWS, GCE, and nginx ingress controllers

Controller Install guide : Here

Installation Guide - Ingress-Nginx Controller

 

 

# Ingress deploy.yaml 다운받기
## Private 환경에 설치했을 경우 Bare-metal로 구성
### 나는 GKE 버전 사용
[master ~]$wget https://raw.githubusercontent.com/kubernetes/ingress-nginx/controller-v1.10.0/deploy/static/provider/cloud/deploy.yaml
--2024-03-07 14:02:59--  https://raw.githubusercontent.com/kubernetes/ingress-nginx/controller-v1.10.0/deploy/static/provider/cloud/deploy.yaml
Resolving raw.githubusercontent.com (raw.githubusercontent.com)... 185.199.108.133, 185.199.109.133, 185.199.110.133, ...
Connecting to raw.githubusercontent.com (raw.githubusercontent.com)|185.199.108.133|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 16136 (16K) [text/plain]
Saving to: ‘deploy.yaml’

deploy.yaml                             100%[==============================================================================>]  15.76K  --.-KB/s    in 0.004s  

2024-03-07 14:03:00 (4.34 MB/s) - ‘deploy.yaml’ saved [16136/16136]

[master ~]$cat deploy.yaml 
apiVersion: v1
kind: Namespace
metadata:
  labels:
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/name: ingress-nginx
  name: ingress-nginx
---
apiVersion: v1
automountServiceAccountToken: true
kind: ServiceAccount              # 인증, 보안 설정을 위한 것
metadata:
  labels:
    app.kubernetes.io/component: controller
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
    app.kubernetes.io/version: 1.10.0
  name: ingress-nginx
  namespace: ingress-nginx
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  labels:
    app.kubernetes.io/component: controller
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
    app.kubernetes.io/version: 1.10.0
  name: ingress-nginx
  namespace: ingress-nginx
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: Role
  name: ingress-nginx
subjects:
- kind: ServiceAccount
  name: ingress-nginx
  namespace: ingress-nginx
--- 
apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    app.kubernetes.io/component: controller
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
    app.kubernetes.io/version: 1.10.0
  name: ingress-nginx-controller
  namespace: ingress-nginx
spec:
  minReadySeconds: 0
  revisionHistoryLimit: 10
  selector:
    matchLabels:
      app.kubernetes.io/component: controller
      app.kubernetes.io/instance: ingress-nginx
      app.kubernetes.io/name: ingress-nginx
  strategy:
    rollingUpdate:
      maxUnavailable: 1
    type: RollingUpdate
  template:
    metadata:
      labels:
        app.kubernetes.io/component: controller
        app.kubernetes.io/instance: ingress-nginx
        app.kubernetes.io/name: ingress-nginx
        app.kubernetes.io/part-of: ingress-nginx
        app.kubernetes.io/version: 1.10.0
    spec:
      containers:
      - args:
        - /nginx-ingress-controller
        - --publish-service=$(POD_NAMESPACE)/ingress-nginx-controller
        - --election-id=ingress-nginx-leader
        - --controller-class=k8s.io/ingress-nginx
        - --ingress-class=nginx
        - --configmap=$(POD_NAMESPACE)/ingress-nginx-controller
        - --validating-webhook=:8443
        - --validating-webhook-certificate=/usr/local/certificates/cert
        - --validating-webhook-key=/usr/local/certificates/key
        - --enable-metrics=false
        env:
        - name: POD_NAME
          valueFrom:
            fieldRef:
              fieldPath: metadata.name
        - name: POD_NAMESPACE
          valueFrom:
            fieldRef:
              fieldPath: metadata.namespace
        - name: LD_PRELOAD
          value: /usr/local/lib/libmimalloc.so
        image: registry.k8s.io/ingress-nginx/controller:v1.10.0@sha256:42b3f0e5d0846876b1791cd3afeb5f1cbbe4259d6f35651dcc1b5c980925379c # 컨트롤러 이미지
        imagePullPolicy: IfNotPresent
        ...
        
# 절차 따라 install 진행
[master ~]$kubectl create clusterrolebinding cluster-admin-binding \
  --clusterrole cluster-admin \
  --user $(gcloud config get-value account)
Your active configuration is: [cloudshell-31728]
clusterrolebinding.rbac.authorization.k8s.io/cluster-admin-binding created

[master ~]$kubectl create -f deploy.yaml 
namespace/ingress-nginx created
serviceaccount/ingress-nginx created
serviceaccount/ingress-nginx-admission created
role.rbac.authorization.k8s.io/ingress-nginx created
role.rbac.authorization.k8s.io/ingress-nginx-admission created
clusterrole.rbac.authorization.k8s.io/ingress-nginx created
clusterrole.rbac.authorization.k8s.io/ingress-nginx-admission created
rolebinding.rbac.authorization.k8s.io/ingress-nginx created
rolebinding.rbac.authorization.k8s.io/ingress-nginx-admission created
clusterrolebinding.rbac.authorization.k8s.io/ingress-nginx created
clusterrolebinding.rbac.authorization.k8s.io/ingress-nginx-admission created
configmap/ingress-nginx-controller created
service/ingress-nginx-controller created
service/ingress-nginx-controller-admission created
deployment.apps/ingress-nginx-controller created
job.batch/ingress-nginx-admission-create created
job.batch/ingress-nginx-admission-patch created
ingressclass.networking.k8s.io/nginx created
validatingwebhookconfiguration.admissionregistration.k8s.io/ingress-nginx-admission created

# ingress-nginx 라는 namespace에서 동작중인 Pod 확인
[master ~]$kubectl get pod -n ingress-nginx
NAME                                        READY   STATUS      RESTARTS   AGE
ingress-nginx-admission-create-v8kkc        0/1     Completed   0          96s
ingress-nginx-admission-patch-8zv6j         0/1     Completed   0          95s
ingress-nginx-controller-654497b5fc-q65w8   1/1     Running     0          96s

# 서비스 확인
[master ~]$kubectl get svc -n ingress-nginx
NAME                                 TYPE           CLUSTER-IP    EXTERNAL-IP     PORT(S)                      AGE
ingress-nginx-controller             LoadBalancer   10.8.11.186   34.30.207.184   80:32581/TCP,443:32714/TCP   2m3s
ingress-nginx-controller-admission   ClusterIP      10.8.2.34     <none>          443/TCP                      2m2s
# GKE라서 LoadBalancer로 controller가 구성이 되었다.
# Private이면 NodePort로 구성된다.

 

# Ingress를 이용한 웹서비스 운영 실습

 

http://www.XXX/  --> svc marvel-service

http://www.XXX/pay --> svc pay-service

 

# ingress namespace 확인
[master ~/ingress]$kubectl get namespaces
NAME              STATUS   AGE
default           Active   9d
gmp-public        Active   9d
gmp-system        Active   9d
ingress-nginx     Active   50m
kube-node-lease   Active   9d
kube-public       Active   9d
kube-system       Active   9d

[master ~/ingress]$kubectl get all -n ingress-nginx
NAME                                            READY   STATUS      RESTARTS   AGE
pod/ingress-nginx-admission-create-v8kkc        0/1     Completed   0          50m
pod/ingress-nginx-admission-patch-8zv6j         0/1     Completed   0          50m
pod/ingress-nginx-controller-654497b5fc-q65w8   1/1     Running     0          50m  ## 동작 중 확인

NAME                                         TYPE           CLUSTER-IP    EXTERNAL-IP     PORT(S)                      AGE
service/ingress-nginx-controller             LoadBalancer   10.8.11.186   34.30.207.184   80:32581/TCP,443:32714/TCP   50m
service/ingress-nginx-controller-admission   ClusterIP      10.8.2.34     <none>          443/TCP                      50m

NAME                                       READY   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/ingress-nginx-controller   1/1     1            1           50m

NAME                                                  DESIRED   CURRENT   READY   AGE
replicaset.apps/ingress-nginx-controller-654497b5fc   1         1         1       50m

NAME                                       COMPLETIONS   DURATION   AGE
job.batch/ingress-nginx-admission-create   1/1           7s         50m
job.batch/ingress-nginx-admission-patch    1/1           7s         50m

따배쿠 강의에서 제공해주는 Hub : https://hub.docker.com/search?q=smlinux

Docker

# namespace를 default에서 ingress로 치환

root@master:~/ingress# kubectl get svc -n ingress-nginx
NAME                                 TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)                      AGE
ingress-nginx-controller             NodePort    10.107.101.97   <none>        80:31821/TCP,443:32755/TCP   49s
ingress-nginx-controller-admission   ClusterIP   10.110.244.50   <none>        443/TCP                      49s
root@master:~/ingress# kubectl get namespaces
NAME              STATUS   AGE
default           Active   21m
ingress-nginx     Active   57s
kube-node-lease   Active   21m
kube-public       Active   21m
kube-system       Active   21m
root@master:~/ingress# kubectl get all -n ingress-nginx
NAME                                           READY   STATUS      RESTARTS   AGE
pod/ingress-nginx-admission-create-mzgm2       0/1     Completed   0          68s
pod/ingress-nginx-admission-patch-pc7f5        0/1     Completed   2          68s
pod/ingress-nginx-controller-5458dd5f6-zkd58   1/1     Running     0          68s

NAME                                         TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)                      AGE
service/ingress-nginx-controller             NodePort    10.107.101.97   <none>        80:31821/TCP,443:32755/TCP   68s
service/ingress-nginx-controller-admission   ClusterIP   10.110.244.50   <none>        443/TCP                      68s
# 31821으로 들어오면 web기반 서비스를 ingress controller가 지원
# 32755로 들어오면 인증서 기반의 웹서비스를 지원해주도록 구성되어 있다.

NAME                                       READY   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/ingress-nginx-controller   1/1     1            1           68s

NAME                                                 DESIRED   CURRENT   READY   AGE
replicaset.apps/ingress-nginx-controller-5458dd5f6   1         1         1       68s

NAME                                       COMPLETIONS   DURATION   AGE
job.batch/ingress-nginx-admission-create   1/1           16s        68s
job.batch/ingress-nginx-admission-patch    1/1           24s        68s


root@master:~/ingress# kubectl config view
apiVersion: v1
clusters:
- cluster:
    certificate-authority-data: DATA+OMITTED
    server: https://10.100.0.104:6443
  name: kubernetes
contexts:
- context:
    cluster: kubernetes
    user: kubernetes-admin
  name: kubernetes-admin@kubernetes
current-context: kubernetes-admin@kubernetes
kind: Config
preferences: {}
users:
- name: kubernetes-admin
  user:
    client-certificate-data: DATA+OMITTED
    client-key-data: DATA+OMITTED

      
      
[master ~]$kubectl config set-context ingress-admin@kubernetes --cluster=kubernetes --user=kubernetes-admin --namespace ingress-nginx
Context "ingress-admin@kubernetes" created.

# context switch
[master ~]$kubectl config use-context ingress-admin@kubernetes
Switched to context "ingress-admin@kubernetes".

# 또는 하기 명령어 사용
[master ~]$kubectl config set-context --current --namespace=ingress-nginx
Context "gke_caramel-element-415606_us-central1-c_cluster-1" modified.

 

# 웹 서비스 동작

[master ~/ingress]$tree marvel-collection/
marvel-collection/
├── Dockerfile
└── html
    ├── images
    │   ├── category.png
    │   └── marvel_logo.png
    └── index.html

2 directories, 4 files

[master ~/ingress]$cat pay.yaml 
apiVersion: v1
kind: ReplicationController
metadata:
  name: pay-rc
spec:
  replicas: 3
  template:
    metadata:
      labels:
        app: pay
    spec:
      containers:
      - image: smlinux/pay
        name: pay
        ports:
        - containerPort: 8080
---
apiVersion: v1
kind: Service
metadata:
  name: pay-service
spec:
  ports:
  - port: 80
    targetPort: 8080
  selector:
    app: pay
    
[master ~/ingress]$cat marvel-home.yaml 
apiVersion: apps/v1
kind: Deployment
metadata:
  name: marvel-home
spec:
  replicas: 1
  selector:
    matchLabels:
      name: marvel
  template:
    metadata:
      labels:
        name: marvel
    spec:
      containers:
      - image: smlinux/marvel-collection
        name: marvel-container
        ports:
        - containerPort: 80
---
apiVersion: v1
kind: Service
metadata:
  name: marvel-service
spec:
  ports:
  - port: 80
    protocol: TCP
    targetPort: 80
  selector:
    name: marvel

    
[master ~/ingress]$kubectl apply -f marvel-home.yaml -f pay.yaml 
deployment.apps/marvel-home created
service/marvel-service created
replicationcontroller/pay-rc created
service/pay-service created

# 서비스 확인
[master ~/ingress]$kubectl get all
NAME                                            READY   STATUS      RESTARTS   AGE
pod/ingress-nginx-admission-create-v8kkc        0/1     Completed   0          92m
pod/ingress-nginx-admission-patch-8zv6j         0/1     Completed   0          92m
pod/ingress-nginx-controller-654497b5fc-q65w8   1/1     Running     0          92m
pod/marvel-home-6b586d7bfc-jpbjw                1/1     Running     0          119s
pod/pay-rc-2f99n                                1/1     Running     0          118s
pod/pay-rc-2ztdc                                1/1     Running     0          118s
pod/pay-rc-zgjb5                                1/1     Running     0          118s

NAME                           DESIRED   CURRENT   READY   AGE
replicationcontroller/pay-rc   3         3         3       119s

NAME                                         TYPE           CLUSTER-IP    EXTERNAL-IP     PORT(S)                      AGE
service/ingress-nginx-controller             LoadBalancer   10.8.11.186   34.30.207.184   80:32581/TCP,443:32714/TCP   92m
service/ingress-nginx-controller-admission   ClusterIP      10.8.2.34     <none>          443/TCP                      92m
service/marvel-service                       ClusterIP      10.8.14.217   <none>          80/TCP                       2m
service/pay-service                          ClusterIP      10.8.2.128    <none>          80/TCP                       119s

NAME                                       READY   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/ingress-nginx-controller   1/1     1            1           92m
deployment.apps/marvel-home                1/1     1            1           2m

NAME                                                  DESIRED   CURRENT   READY   AGE
replicaset.apps/ingress-nginx-controller-654497b5fc   1         1         1       92m
replicaset.apps/marvel-home-6b586d7bfc                1         1         1       2m

NAME                                       COMPLETIONS   DURATION   AGE
job.batch/ingress-nginx-admission-create   1/1           7s         92m
job.batch/ingress-nginx-admission-patch    1/1           7s         92m

[master ~/ingress]$cat ingress_new.yaml 
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: marvel-ingress
  namespace: ingress-nginx
  annotations:
    kubernetes.io/ingress.class: nginx
spec:
  defaultBackend:
    service:
      name: nginx
      port:
        number: 80
  rules:
  - host: node1.example.com
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: marvel-service
            port:
              number: 80
  - host: node2.example.com
    http:
      paths:
      - path: /pay
        pathType: Prefix
        backend:
          service:
            name: pay-service
            port:
              number: 80
              
[master ~/ingress]$kubectl create -f ingress_new.yaml               
Warning: annotation "kubernetes.io/ingress.class" is deprecated, please use 'spec.ingressClassName' instead
ingress.networking.k8s.io/marvel-ingress created

[master ~/ingress]$kubectl get ingress
NAME             CLASS    HOSTS                                 ADDRESS         PORTS   AGE
marvel-ingress   <none>   node1.example.com,node2.example.com   34.30.207.184   80      75s

[master ~/ingress]$kubectl describe ingress marvel-ingress
Name:             marvel-ingress
Labels:           <none>
Namespace:        ingress-nginx
Address:          34.30.207.184
Ingress Class:    <none>
Default backend:  nginx:80 (<error: endpoints "nginx" not found>)
Rules:
  Host               Path  Backends
  ----               ----  --------
  node1.example.com  
                     /   marvel-service:80 (10.4.0.23:80)
  node2.example.com  
                     /pay   pay-service:80 (10.4.0.24:8080,10.4.1.15:8080,10.4.2.9:8080)
Annotations:         kubernetes.io/ingress.class: nginx
Events:
  Type    Reason  Age                From                      Message
  ----    ------  ----               ----                      -------
  Normal  Sync    20s (x2 over 61s)  nginx-ingress-controller  Scheduled for sync