duiniwukenaihe SRE Engineer(target)

traefik2 安装实现 http https

2019-10-17
duiniwukenaihe

描述背景:

注: 文章配置文件示例都是拿阳明大佬的还有良哥转发的http://www.mydlq.club/article/41 超级小豆丁的文章中参考的。个人只是拿来整合测试了下整合,鸣谢各位大佬。

开始使用腾讯云tke,slb对外映射方式 还需要对外开启Nodeport 然后slb对端口映射,个人觉得比较反人类,然后看到阳明 良哥两位大佬都发了traefik的方式就试用了一下。

集群配置:
初始集群环境kubeadm 1.16.1

|  ip           | 自定义域名         |    主机名 |
|  :----:       |     :----:        |   :----:  |
|192.168.3.8      |  master.k8s.io    |  k8s-vip  |
|192.168.3.10    |  master01.k8s.io  |  k8s-master-01|
|192.168.3.5   |  master02.k8s.io  |  k8s-master-02| 
|192.168.3.12   |  master03.k8s.io  |  k8s-master-03|
|192.168.3.6    |  node01.k8s.io    |  k8s-node-01|
|192.168.3.2    |  node02.k8s.io    |  k8s-node-02|
|192.168.3.4    |  node03.k8s.io    |  k8s-node-03|
k8s-node-01 k8s-node-02 k8s-node-03 绑定了腾讯云负载均衡应用负载均衡并对外暴露了80  443两个端口,开始443想放在https监听器哪里,但是这样一个sbl负载均衡只能绑定一个证书的,业务不太多,不想用多个slb,就将443端口监听放在了tcp/udp监听器这里,然后证书在kubernetes集群中创建绑定在traefik上。

关于Traefik

Traefik 2.0 官方文档:https://docs.traefik.io/v2.0/ 比较乱,中文的可参照阳明大佬的中文翻译https://www.qikqiak.com/traefik-book/

Kubernetes 部署 Traefik

1. 创建CRD资源

cat <<EOF >  traefik-crd.yaml 
## IngressRoute
apiVersion: apiextensions.k8s.io/v1beta1
kind: CustomResourceDefinition
metadata:
  name: ingressroutes.traefik.containo.us
spec:
  scope: Namespaced
  group: traefik.containo.us
  version: v1alpha1
  names:
    kind: IngressRoute
    plural: ingressroutes
    singular: ingressroute
---
## IngressRouteTCP
apiVersion: apiextensions.k8s.io/v1beta1
kind: CustomResourceDefinition
metadata:
  name: ingressroutetcps.traefik.containo.us
spec:
  scope: Namespaced
  group: traefik.containo.us
  version: v1alpha1
  names:
    kind: IngressRouteTCP
    plural: ingressroutetcps
    singular: ingressroutetcp
---
## Middleware
apiVersion: apiextensions.k8s.io/v1beta1
kind: CustomResourceDefinition
metadata:
  name: middlewares.traefik.containo.us
spec:
  scope: Namespaced
  group: traefik.containo.us
  version: v1alpha1
  names:
    kind: Middleware
    plural: middlewares
    singular: middleware
---
apiVersion: apiextensions.k8s.io/v1beta1
kind: CustomResourceDefinition
metadata:
  name: tlsoptions.traefik.containo.us
spec:
  scope: Namespaced
  group: traefik.containo.us
  version: v1alpha1
  names:
    kind: TLSOption
    plural: tlsoptions
    singular: tlsoption
EOF
kubectl apply -f traefik-crd.yaml

2. 创建RBAC权限

cat <<EOF > traefik-rbac.yaml
apiVersion: v1
kind: ServiceAccount
metadata:
  namespace: kube-system
  name: traefik-ingress-controller
---
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
  name: traefik-ingress-controller
rules:
  - apiGroups: [""]
    resources: ["services","endpoints","secrets"]
    verbs: ["get","list","watch"]
  - apiGroups: ["extensions"]
    resources: ["ingresses"]
    verbs: ["get","list","watch"]
  - apiGroups: ["extensions"]
    resources: ["ingresses/status"]
    verbs: ["update"]
  - apiGroups: ["traefik.containo.us"]
    resources: ["middlewares"]
    verbs: ["get","list","watch"]
  - apiGroups: ["traefik.containo.us"]
    resources: ["ingressroutes"]
    verbs: ["get","list","watch"]
  - apiGroups: ["traefik.containo.us"]
    resources: ["ingressroutetcps"]
    verbs: ["get","list","watch"]
  - apiGroups: ["traefik.containo.us"]
    resources: ["tlsoptions"]
    verbs: ["get","list","watch"]
---
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
  name: traefik-ingress-controller
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: traefik-ingress-controller
subjects:
  - kind: ServiceAccount
    name: traefik-ingress-controller
    namespace: kube-system
EOF
 kubectl apply -f traefik-rbac.yaml 

3.创建 Traefik 配置文件

cat <<EOF > traefik-config.yaml
kind: ConfigMap
apiVersion: v1
metadata:
  namespace: kube-system
  name: traefik-config
data:
  traefik.yaml: |-
    serversTransport:
      insecureSkipVerify: true
    api:
      insecure: true
      dashboard: true
      debug: true
    metrics:
      prometheus: ""
    entryPoints:
      web:
        address: ":80"
      websecure:
        address: ":443"
    providers:
      kubernetesCRD: ""
    log:
      filePath: ""
      level: error
      format: json
    accessLog:
      filePath: ""
      format: json
      bufferingSize: 0
      filters:
        retryAttempts: true
        minDuration: 20
      fields:
        defaultMode: keep
        names:
          ClientUsername: drop
        headers:
          defaultMode: keep
          names:
            User-Agent: redact
            Authorization: drop
            Content-Type: keep

EOF
kubectl apply -f traefik-config.yaml 

4、节点设置 Label 标签

Traefix 采用 DaemonSet方式构建,在需要安装的节点上面打上标签,这里在三个work节点都安装上了默认:

kubectl label nodes {k8s-node-01,k8s-node-02,k8s-node-03} IngressProxy=true

kubectl get nodes --show-labels

lables.png

注意:如果想删除标签,可以使用 kubectl label nodes k8s-node-03 IngressProxy- 命令

5. Kubernetes 部署 Traefik

cat <<EOF > traefik-deploy.yaml
apiVersion: v1
kind: Service
metadata:
  namespace: kube-system
  name: traefik
spec:
  ports:
    - name: web
      port: 80
    - name: websecure
      port: 443
    - name: admin
      port: 8080
  selector:
    app: traefik
---
apiVersion: apps/v1
kind: DaemonSet
metadata:
  namespace: kube-system
  name: traefik-ingress-controller
  labels:
    app: traefik
spec:
  selector:
    matchLabels:
      app: traefik
  template:
    metadata:
      name: traefik
      labels:
        app: traefik
    spec:
      serviceAccountName: traefik-ingress-controller
      terminationGracePeriodSeconds: 1
      containers:
        - image: traefik:latest
          name: traefik-ingress-lb
          ports:
            - name: web
              containerPort: 80
              hostPort: 80           #hostPort方式,将端口暴露到集群节点
            - name: websecure
              containerPort: 443
              hostPort: 443          #hostPort方式,将端口暴露到集群节点
            - name: admin
              containerPort: 8080
          resources:
            limits:
              cpu: 2000m
              memory: 1024Mi
            requests:
              cpu: 1000m
              memory: 1024Mi
          securityContext:
            capabilities:
              drop:
                - ALL
              add:
                - NET_BIND_SERVICE
          args:
            - --configfile=/config/traefik.yaml
          volumeMounts:
            - mountPath: "/config"
              name: "config"
      volumes:
        - name: config
          configMap:
            name: traefik-config 
      tolerations:              #设置容忍所有污点,防止节点被设置污点
        - operator: "Exists"
      nodeSelector:             #设置node筛选器,在特定label的节点上启动
        IngressProxy: "true"
EOF
kubectl apply -f traefik-deploy.yaml 

Traefik 路由规则配置

在上一步Kubernetes 部署 Traefik 时候开启了 Traefik Dashboard,现在将traefik-dashboard-route对外暴露。
http的方式就不列举了直接https了 首先 建立https用的secret:

kubectl create secret tls all-saynaihe-com --key=2_saynaihe.com.key --cert=1_saynaihe.com_bundle.crt  -n kube-system

key.png 支持https:

cat <<EOF> traefik-dashboard-route-https.yaml
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:
  namespace: kube-system
  name: traefik-dashboard-route-https
spec:
  entryPoints:
    - websecure
  tls:
    secretName: all-saynaihe-com
  routes:
    - match: Host(\`traefik.saynaihe.com\`)
      kind: Rule
      services:
        - name: traefik
          port: 8080

EOF
kubectl apply -f traefik-dashboard-route-https.yaml

traefik.png

再来个支持http的

cat <<EOF> traefik-dashboard-route-http.yaml

apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:
  namespace: kube-system
  name: traefik-dashboard-route-http
spec:
  entryPoints:
    - web
  routes:
    - match: Host(\`traefik.saynaihe.com\`)
      kind: Rule
      services:
        - name: traefik
          port: 8080
EOF
kubectl apply -f traefik-dashboard-route-http.yaml

traefik-http.png traefik-http1.png route.png

然后是否需要http 强调https,添加一个 Middleware 的 CRD 对象

cat <<EOF> redirect-https.yaml
apiVersion: traefik.containo.us/v1alpha1
kind: Middleware
metadata:
  name: redirect-https
  namespace: kube-system
spec:
  redirectScheme:
    scheme: https
EOF
kubectl apply -f redirect-https.yaml
---

cat <<EOF> traefik-dashboard-route.yaml
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:
  namespace: kube-system
  name: redirect-https
  name: traefik-dashboard-route
spec:
  entryPoints:
    - web
  routes:
    - match: Host(\`traefik.saynaihe.com\`)
      kind: Rule
      services:
        - name: traefik
          port: 8080
      middlewares:  # 使用上面新建的中间件
        - name: redirect-https

EOF
kubectl apply -f traefik-dashboard-route.yaml

通过测试 http强跳https了 ,tcp的下次写了呢。


Similar Posts

Comments