描述背景:
注: 文章配置文件示例都是拿阳明大佬的还有良哥转发的http://www.mydlq.club/article/41 超级小豆丁的文章中参考的。个人只是拿来整合测试了下整合,鸣谢各位大佬。
开始使用腾讯云tke,slb对外映射方式 还需要对外开启Nodeport 然后slb对端口映射,个人觉得比较反人类,然后看到阳明 良哥两位大佬都发了traefik的方式就试用了一下。
集群配置:
初始集群环境kubeadm 1.16.1
| ip | 自定义域名 | 主机名 |
| :----: | :----: | :----: |
|192.168.3.8 | master.k8s.io | k8s-vip |
|192.168.3.10 | master01.k8s.io | k8s-master-01|
|192.168.3.5 | master02.k8s.io | k8s-master-02|
|192.168.3.12 | master03.k8s.io | k8s-master-03|
|192.168.3.6 | node01.k8s.io | k8s-node-01|
|192.168.3.2 | node02.k8s.io | k8s-node-02|
|192.168.3.4 | node03.k8s.io | k8s-node-03|
k8s-node-01 k8s-node-02 k8s-node-03 绑定了腾讯云负载均衡应用负载均衡并对外暴露了80 443两个端口,开始443想放在https监听器哪里,但是这样一个sbl负载均衡只能绑定一个证书的,业务不太多,不想用多个slb,就将443端口监听放在了tcp/udp监听器这里,然后证书在kubernetes集群中创建绑定在traefik上。
关于Traefik
Traefik 2.0 官方文档:https://docs.traefik.io/v2.0/ 比较乱,中文的可参照阳明大佬的中文翻译https://www.qikqiak.com/traefik-book/
Kubernetes 部署 Traefik
1. 创建CRD资源
cat <<EOF > traefik-crd.yaml
## IngressRoute
apiVersion: apiextensions.k8s.io/v1beta1
kind: CustomResourceDefinition
metadata:
name: ingressroutes.traefik.containo.us
spec:
scope: Namespaced
group: traefik.containo.us
version: v1alpha1
names:
kind: IngressRoute
plural: ingressroutes
singular: ingressroute
---
## IngressRouteTCP
apiVersion: apiextensions.k8s.io/v1beta1
kind: CustomResourceDefinition
metadata:
name: ingressroutetcps.traefik.containo.us
spec:
scope: Namespaced
group: traefik.containo.us
version: v1alpha1
names:
kind: IngressRouteTCP
plural: ingressroutetcps
singular: ingressroutetcp
---
## Middleware
apiVersion: apiextensions.k8s.io/v1beta1
kind: CustomResourceDefinition
metadata:
name: middlewares.traefik.containo.us
spec:
scope: Namespaced
group: traefik.containo.us
version: v1alpha1
names:
kind: Middleware
plural: middlewares
singular: middleware
---
apiVersion: apiextensions.k8s.io/v1beta1
kind: CustomResourceDefinition
metadata:
name: tlsoptions.traefik.containo.us
spec:
scope: Namespaced
group: traefik.containo.us
version: v1alpha1
names:
kind: TLSOption
plural: tlsoptions
singular: tlsoption
EOF
kubectl apply -f traefik-crd.yaml
2. 创建RBAC权限
cat <<EOF > traefik-rbac.yaml
apiVersion: v1
kind: ServiceAccount
metadata:
namespace: kube-system
name: traefik-ingress-controller
---
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
name: traefik-ingress-controller
rules:
- apiGroups: [""]
resources: ["services","endpoints","secrets"]
verbs: ["get","list","watch"]
- apiGroups: ["extensions"]
resources: ["ingresses"]
verbs: ["get","list","watch"]
- apiGroups: ["extensions"]
resources: ["ingresses/status"]
verbs: ["update"]
- apiGroups: ["traefik.containo.us"]
resources: ["middlewares"]
verbs: ["get","list","watch"]
- apiGroups: ["traefik.containo.us"]
resources: ["ingressroutes"]
verbs: ["get","list","watch"]
- apiGroups: ["traefik.containo.us"]
resources: ["ingressroutetcps"]
verbs: ["get","list","watch"]
- apiGroups: ["traefik.containo.us"]
resources: ["tlsoptions"]
verbs: ["get","list","watch"]
---
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
name: traefik-ingress-controller
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: traefik-ingress-controller
subjects:
- kind: ServiceAccount
name: traefik-ingress-controller
namespace: kube-system
EOF
kubectl apply -f traefik-rbac.yaml
3.创建 Traefik 配置文件
cat <<EOF > traefik-config.yaml
kind: ConfigMap
apiVersion: v1
metadata:
namespace: kube-system
name: traefik-config
data:
traefik.yaml: |-
serversTransport:
insecureSkipVerify: true
api:
insecure: true
dashboard: true
debug: true
metrics:
prometheus: ""
entryPoints:
web:
address: ":80"
websecure:
address: ":443"
providers:
kubernetesCRD: ""
log:
filePath: ""
level: error
format: json
accessLog:
filePath: ""
format: json
bufferingSize: 0
filters:
retryAttempts: true
minDuration: 20
fields:
defaultMode: keep
names:
ClientUsername: drop
headers:
defaultMode: keep
names:
User-Agent: redact
Authorization: drop
Content-Type: keep
EOF
kubectl apply -f traefik-config.yaml
4、节点设置 Label 标签
Traefix 采用 DaemonSet方式构建,在需要安装的节点上面打上标签,这里在三个work节点都安装上了默认:
kubectl label nodes {k8s-node-01,k8s-node-02,k8s-node-03} IngressProxy=true
kubectl get nodes --show-labels
注意:如果想删除标签,可以使用 kubectl label nodes k8s-node-03 IngressProxy- 命令
5. Kubernetes 部署 Traefik
cat <<EOF > traefik-deploy.yaml
apiVersion: v1
kind: Service
metadata:
namespace: kube-system
name: traefik
spec:
ports:
- name: web
port: 80
- name: websecure
port: 443
- name: admin
port: 8080
selector:
app: traefik
---
apiVersion: apps/v1
kind: DaemonSet
metadata:
namespace: kube-system
name: traefik-ingress-controller
labels:
app: traefik
spec:
selector:
matchLabels:
app: traefik
template:
metadata:
name: traefik
labels:
app: traefik
spec:
serviceAccountName: traefik-ingress-controller
terminationGracePeriodSeconds: 1
containers:
- image: traefik:latest
name: traefik-ingress-lb
ports:
- name: web
containerPort: 80
hostPort: 80 #hostPort方式,将端口暴露到集群节点
- name: websecure
containerPort: 443
hostPort: 443 #hostPort方式,将端口暴露到集群节点
- name: admin
containerPort: 8080
resources:
limits:
cpu: 2000m
memory: 1024Mi
requests:
cpu: 1000m
memory: 1024Mi
securityContext:
capabilities:
drop:
- ALL
add:
- NET_BIND_SERVICE
args:
- --configfile=/config/traefik.yaml
volumeMounts:
- mountPath: "/config"
name: "config"
volumes:
- name: config
configMap:
name: traefik-config
tolerations: #设置容忍所有污点,防止节点被设置污点
- operator: "Exists"
nodeSelector: #设置node筛选器,在特定label的节点上启动
IngressProxy: "true"
EOF
kubectl apply -f traefik-deploy.yaml
Traefik 路由规则配置
在上一步Kubernetes 部署 Traefik 时候开启了 Traefik Dashboard,现在将traefik-dashboard-route对外暴露。
http的方式就不列举了直接https了 首先 建立https用的secret:
kubectl create secret tls all-saynaihe-com --key=2_saynaihe.com.key --cert=1_saynaihe.com_bundle.crt -n kube-system
支持https:
cat <<EOF> traefik-dashboard-route-https.yaml
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:
namespace: kube-system
name: traefik-dashboard-route-https
spec:
entryPoints:
- websecure
tls:
secretName: all-saynaihe-com
routes:
- match: Host(\`traefik.saynaihe.com\`)
kind: Rule
services:
- name: traefik
port: 8080
EOF
kubectl apply -f traefik-dashboard-route-https.yaml
再来个支持http的
cat <<EOF> traefik-dashboard-route-http.yaml
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:
namespace: kube-system
name: traefik-dashboard-route-http
spec:
entryPoints:
- web
routes:
- match: Host(\`traefik.saynaihe.com\`)
kind: Rule
services:
- name: traefik
port: 8080
EOF
kubectl apply -f traefik-dashboard-route-http.yaml
然后是否需要http 强调https,添加一个 Middleware 的 CRD 对象
cat <<EOF> redirect-https.yaml
apiVersion: traefik.containo.us/v1alpha1
kind: Middleware
metadata:
name: redirect-https
namespace: kube-system
spec:
redirectScheme:
scheme: https
EOF
kubectl apply -f redirect-https.yaml
---
cat <<EOF> traefik-dashboard-route.yaml
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:
namespace: kube-system
name: redirect-https
name: traefik-dashboard-route
spec:
entryPoints:
- web
routes:
- match: Host(\`traefik.saynaihe.com\`)
kind: Rule
services:
- name: traefik
port: 8080
middlewares: # 使用上面新建的中间件
- name: redirect-https
EOF
kubectl apply -f traefik-dashboard-route.yaml
通过测试 http强跳https了 ,tcp的下次写了呢。