监控相关 - Sim Wiki

最后修改人	sim
最后修改时间	2022-03-25 17:45:13
创建人	sim
创建时间	2022-03-24 18:41:29

该文档由 sim 创建于 2022-03-24 18:41:29 ， sim 在 2022-03-25 17:45:13 修改了该文档。

[TOCM]

# Pixie介绍
Pixie是基于Ebpf技术构建的一套可观测性平台，Pixie默认已经集成了许多可观测性工具，例如：我们可以清晰的通过Pixie观测到K8S内部的流量情况、DNS解析时延、TCP丢包、掉包等，同时我们还可以通过Pixie多K8S内Namespace、Pod等进行监控。

Pixie官方主要给出了两种搭建模式：
1. Community Cloud for Pixie 只部署采集端，服务端使用社区提供的公有云
1. Self-Hosted模式 完整的采集端与服务端本地部署

环境要求
Pixie对于部署环境要求有：
```shell
Linux内核版本 >= 4.10
Kubernetes >= 1.16
Kubelet >= 1.10
```

# 升级内核
```shell
cd /usr/local/src
wget http://193.49.22.109/elrepo/kernel/el7/x86_64/RPMS/kernel-ml-4.19.12-1.el7.elrepo.x86_64.rpm
wget http://193.49.22.109/elrepo/kernel/el7/x86_64/RPMS/kernel-ml-devel-4.19.12-1.el7.elrepo.x86_64.rpm
yum localinstall -y kernel-ml-*.rpm
grub2-set-default 0 && grub2-mkconfig -o /boot/grub2/grub.cfg
reboot
```
加载mod_br，关闭swap
```shell
modprobe br_netfilter
cat <<EOF | sudo tee /etc/sysctl.d/k8s.conf
net.ipv4.ip_forward = 1
net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1
EOF
sudo sysctl --system
swapoff -a
```

# 安装 Kubernetes,Docker

安装Docker,并修改daemon.json
```shell
yum install -y yum-utils device-mapper-persistent-data lvm2
yum-config-manager --add-repo http://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo
sed -i 's+download.docker.com+mirrors.aliyun.com/docker-ce+' /etc/yum.repos.d/docker-ce.repo
yum makecache fast
yum install -y docker-ce-20.10.8-3.el7
#启动docker并设置开机启动
systemctl enable --now docker
```
修改daemon.json
```shell
vi /etc/docker/daemon.json
#添加
{
"exec-opts": ["native.cgroupdriver=systemd"],
"registry-mirrors": ["https://pzr80u0u.mirror.aliyuncs.com"]
}
#重启docker
systemctl daemon-reload && systemctl restart docker
```
安装Kubeadm，Kubelet，Kubectl
```shell
cat <<EOF > /etc/yum.repos.d/kubernetes.repo
[kubernetes]
name=Kubernetes
baseurl=https://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64/
enabled=1
gpgcheck=1
repo_gpgcheck=1
gpgkey=https://mirrors.aliyun.com/kubernetes/yum/doc/yum-key.gpg https://mirrors.aliyun.com/kubernetes/yum/doc/rpm-package-key.gpg
EOF
setenforce 0
yum install -y kubelet kubeadm kubectl
#or gpg检查失败用下面命令
yum install -y --nogpgcheck kubelet kubeadm kubectl
systemctl enable kubelet && systemctl start kubelet
```
ps: 由于官网未开放同步方式, 可能会有索引gpg检查失败的情况, 这时请用 
`yum install -y --nogpgcheck kubelet kubeadm kubectl` 安装

部署K8S集群
```shell
#创建kubeadm配置文件模板
kubeadm config print init-defaults > kubeadm.yaml
#修改kubeadm配置文件
vim kubeadm.yaml
##
advertiseAddress: 172.27.18.19              #本机内网IP地址
name: cka-sim-master                        #修改master的集群名，需要在hosts中指定
imageRepository: registry.cn-hangzhou.aliyuncs.com/google_containers    #修改下载镜像源
```
创建集群
```shell
kubeadm init --config kubeadm.yaml
#完成后设置环境变量：
mkdir -p $HOME/.kube
cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
chown $(id -u):$(id -g) $HOME/.kube/config
#export添加至/etc/profile后source加载
export KUBECONFIG=/etc/kubernetes/admin.conf
```
安装Calico网络插件
```shell
wget --no-check-certificate https://docs.projectcalico.org/manifests/calico.yaml
kubectl apply -f calico.yaml
```
添加命令补全
```shell
yum install -y bash-completion
source /usr/share/bash-completion/bash_completion
source <(kubectl completion bash)
```
Node节点加入集群
```shell
#在Master节点输入
kubeadm token create --print-join-command
#复制生成命令去Node节点执行
kubeadm join 172.27.18.19:6443 --token 9mnlx9.8lfuswjb57u289fm --discovery-token-ca-cert-hash sha256:7c33b38f36a09ab1fc250713203d6ad2e46ceaee9b1efc670f2acc2f65e95581 
```
去除Master污点，用于参加调度
```shell
kubectl taint node k8s-master-01 node-role.kubernetes.io/master-
```

# 安装Pixie
[Pixie官方文档](https://docs.px.dev/installing-pixie/install-guides/self-hosted-pixie/ "Pixie官方文档")
首先下载代码：
```shell
cd /home
git clone https://github.com/pixie-io/pixie.git
cd pixie
```
安装brew
```shell
cd /home
git clone https://github.com/Homebrew/brew homebrew
eval "$(/home/homebrew/bin/brew shellenv)"
brew update --force --quiet
chmod -R go-w "$(brew --prefix)/share/zsh"
```
安装mkcert
```shell
yum install -y nss-tools
brew install -f --ignore-dependencies mkcert
# 将安装完的mkcert路径添加到PATH中
# /home/homebrew/Cellar/mkcert/1.4.3/bin
mkcert --install
```
配置Namespaces
```shell
kubectl create namespace plc
cd $(pixie)
./scripts/create_cloud_secrets.sh
```
安装kustomize
```shell
curl -s "https://raw.githubusercontent.com/\
kubernetes-sigs/kustomize/master/hack/install_kustomize.sh"  | bash
ln -s /home/pixie/kustomize /usr/bin/
```
部署Pixie Cloud
```shell
kustomize build k8s/cloud_deps/base/elastic/operator | kubectl apply -f -
kustomize build k8s/cloud_deps/public | kubectl apply -f -
kustomize build k8s/cloud/public/ | kubectl apply -f -
kubectl -n plx get pods
```