分享交流
合作共赢!

Kubetools - A Curated List of Kubernetes Tools

张琼杰阅读(503)

There are more than 300+ Kubernetes Certified Service Providers and tons of Kubernetes Certified distributions. Choosing a right distribution can be a daunting task. Kubetools is built with a purpose to build a curated list of popular Kubernetes tools. It is actively maintained by Collabnix Slack Community.

Follow the Kubetools Twitter account for updates on new list additions.

Have Questions? Join us over Slack and get chance to be a part of 7200+ DevOps enthusiasts.

Cluster Management

kops – Production Grade K8s Installation, Upgrades, and Management

silver-surfer – Check ApiVersion compatibility and provide Migration path for Kubernetes objects when upgrading Kubernetes to latest versions

Kube-ops-view – Kubernetes Operational View – read-only system dashboard for multiple K8s clusters

kubeprompt – Kubernetes prompt info

Metalk8s – An opinionated Kubernetes distribution with a focus on long-term on-prem deployments

kind – Kubernetes IN Docker – local clusters for testing Kubernetes

Clusterman – Cluster Autoscaler for Kubernetes and Mesos

Cert-manager – Automatically provision and manage TLS certificates

Goldilocks – Get your resource requests “Just Right”

katafygio – Dump, or continuously backup Kubernetes objets as yaml files in git

Rancher – Complete container management platform

Sealed Secrets – A Kubernetes controller and tool for one-way encrypted Secrets

OpenKruise/Kruise – Automate application workloads management on Kubernetes https://openkruise.io

kubectl snapshot – Take Cluster Snapshots

kapp – simple deployment tool focused on the concept of “Kubernetes application” — a set of resources with the same label https://get-kapp.io

keda – Event-driven autoscaler for Kubernetes

Octant – To better understand the complexity of Kubernetes clusters

Portainer – Portainer inside a Kubernetes environment

Gardener – Deliver fully-managed clusters at scale everywhere with your own Kubernetes-as-a-Service

xlskubectl – xlskubectl — a spreadsheet to control your Kubernetes cluster

Cluster with Core CLI tools

Bootkube – bootkube – Launch a self-hosted Kubernetes cluster

kubectx + kubens – Switch faster between clusters and namespaces in kubectl

kube-shell – Kubernetes shell: An integrated shell for working with the Kubernetes

kuttle: kubectl wrapper for sshuttle without SSH – Kubernetes wrapper for sshuttle

kubectl sudo – Run kubernetes commands with the security privileges of another user

K9s – Kubernetes CLI To Manage Your Clusters In Style!

Ktunnel – A cli that exposes your local resources to kubernetes

KubeOperator – Run kubectl command in Web Browser. https://kubeoperator.io/

Vimkubectl – Manage any Kubernetes resource from Vim https://www.vim.org/scripts/script.ph

KubeHelper – KubeHelper – simplifies many daily Kubernetes cluster tasks through a web interface.

Alert and Monitoring

Thanos – Highly available Prometheus setup with long term storage capabilities. CNCF Sandbox project. https://thanos.io

Prometheus – The Prometheus monitoring system and time series database.

Grafana – The tool for beautiful monitoring and metric analytics & dashboards for Graphite, InfluxDB & Prometheus & More

Kubetail – Bash script to tail Kubernetes logs from multiple pods at the same time

Searchlight – Alerts for Kubernetes

linkerd2 Monitoring Mixin for Grafana – Grafana dashboards for linkerd2 monitoring and can work in standalone (default) or in multi cluster setup

kuberhaus – Kubernetes resource dashboard with node/pod layout and resource requests

Kubernetes Job/CronJob Notifier – This tool sends an alert to slack whenever there is a Kubernetes cronJob/Job failure/success

Argus – This tool monitors changes in the filesystem on specified paths

Kube-Scout – Scout for alarming issues across your Kubernetes clusters

Logging and Tracing

Jaeger – CNCF Jaeger, a Distributed Tracing Platform

Kiali – Kiali project, observability for the Istio service mesh

ELK – Elasticsearch, Logstash, Kibana

fluentbit – Fast and Lightweight Log processor and forwarder for Linux, BSD and OSX

Loki – Like Prometheus, but for logs

Troubleshooting

Kubectl-debug – Allows you to run a new container with all the troubleshooting tools installed in running pod for debugging purposed

PowerfulSeal – A powerful testing tool for Kubernetes clustersd

Crash-diagnostic – Crash-Diagnostics is a tool to help investigate, analyze, and troubleshoot unresponsive or crashed Kubernetes clustersd

K9s – Kubernetes CLI To Manage Your Clusters In Style!d

Kubernetes CLI Plugin – Doctor – kubectl cluster triage plugin for k8s – 🏥 (brew doctor equivalent)d

Knative Inspect – A light-weight debugging tool for Knative’s system componentsd

Kubeman – To find information from Kubernetes clusters, and to investigate issues related to Kubernetes and Istiod

kpexec – kpexec is a kubernetes cli that runs commands in a container with high privilegesd

Koolkits – 🧰 Opinionated, language-specific, batteries-included debug container images for Kubernetes.

Developement Tools/Kit

Okteto: A Tool for Cloud Native Developers – Build better applications by developing and testing your code directly in Kubernetes

Tilt: Tilt manages local development instances for teams that deploy to Kubernetes – Local Kubernetes development with no stress

Garden: Kubernetes from source to finish – Development orchestrator for Kubernetes, containers and functions.

KuberNix – Single dependency Kubernetes clusters for local testing, experimenting and development

Copper – A configuration file validator for Kubernetes

ko – Build and deploy Go applications on Kubernetes

Dekorate – Java annotation processors for Kubernetes

Lens IDE – A powerful interface and toolkit for managing, visualizing, and interacting with multiple Kubernetes clusters

Kosko – Organize Kubernetes manifests in JavaScript

Telepresence – Fast, local development for Kubernetes and Openshift microservices

Monokle – Desktop UI for managing Kubernetes manifests

KuberEz – Graphical modeling tool for Kubernetes manifest

mirrord – Run your local process in the context of your cloud cluster

Alternative Tools for Developement

Minikube – minikube implements a local Kubernetes clusterd

KubeSphere – Easy-to-use Production Ready Container Platform https://kubesphere.io

skippbox – A Desktop application for k8sd

kind – Kubernetes IN Docker – local clusters for testing Kubernetes https://kind.sigs.k8s.io/d

k3d – k3d is a lightweight wrapper to run k3s (Rancher Lab’s minimal Kubernetes distribution) in docker.d

Systemk: virtual kubelet for systemd – Systemk is a systemd backend for the virtual-kubelet. Instead of starting containers, you start systemd units

CI/CD integration Tools

HybridK8s Droid – Intelligence foor your favourite Delivery Platform

Devtron – Software Delivery Workflow for Kubernetes

Skaffold – Easy and Repeatable Kubernetes Development

Apollo – Apollo – The logz.io continuous deployment solution over kubernetes

Helm Cabin – Web UI that visualizes Helm releases in a Kubernetes cluster

flagger – Progressive delivery Kubernetes operator (Canary, A/B Testing and Blue/Green deployments)

Kubeform – Kubernetes CRDs for Terraform providers https://kubeform.com

Spinnaker – Spinnaker is an open source, multi-cloud continuous delivery platform for releasing software changes with high velocity and confidence. http://www.spinnaker.io/

werf – GitOps tool to deliver apps to Kubernetes and integrate this process with GitLab and other CI tools

Flux – GitOps Kubernetes operator

Argo CD – Declarative continuous deployment for Kubernetes

Tekton – A cloud native continuous integration and delivery (CI/CD) solution

Jenkins X – Jenkins X provides automated CI+CD for Kubernetes with Preview Environments on Pull Requests using Tekton, Knative, Lighthouse, Skaffold and Helm

Drone – Drone is a Container-Native, Continuous Delivery Platform

Security Tools

TerraScan – Detect compliance and security violations across Infrastructure as Code to mitigate risk before provisioning cloud native infrastructure.

klum – Kubernetes Lazy User Manager

kube2iam – IAM credentials to containers running inside a kubernetes cluster based on annotations.

Kyverno – Kubernetes Native Policy Management https://kyverno.io

kiosk – kiosk office Multi-Tenancy Extension For Kubernetes – Secure Cluster Sharing & Self-Service Namespace Provisioning

kube-bench – CIS Kubernetes Benchmark tool

kube-hunter – Pentesting tool – Hunts for security weaknesses in Kubernetes clusters

kube-who-can – Show who has RBAC permissions to perform actions on different resources in Kubernetes

starboard – Kubernetes-native security toolkit

Simulator – Kubernetes Security Training Platform – Focussing on security mitigation

RBAC Lookup – Easily find roles and cluster roles attached to any user, service account, or group name in your Kubernetes cluster https://fairwinds.com

Kubeaudit – kubeaudit helps you audit your Kubernetes clusters against common security controls

Gangway – An application that can be used to easily enable authentication flows via OIDC for a kubernetes cluster

Audit2rbac – Autogenerate RBAC policies based on Kubernetes audit logs

Chartsec – Helm Chart security scanner

kubestriker – Security Auditing tool

Datree – CLI tool to prevent K8s misconfigurations by ensuring that manifests and Helm charts follow best practices as well as your organization’s policies

Krane – Kubernetes RBAC static Analysis & visualisation tool

Falco – The Falco Project – Cloud-Native runtime security

Clair – Vulnerability Static Analysis for Containers

Network Policies

trireme-kubernetes – Aporeto integration with Kubernetes Network Policies

Calico – Cloud native connectivity and network policy

kubepox – Kubernetes network Policy eXploration tool

kokotap – Tools for kubernetes pod network tapping

Submariner – Connect all your Kubernetes clusters, no matter where they are in the world

egress-operator – An operator to produce egress gateway pods and control access to them with network policies

kubefwd (Kube Forward) – Bulk port forwarding Kubernetes services for local development

Testing Tools

k6d – A modern load testing tool, using Go and JavaScript

Network bandwith and load testingd – Test suite for Kubernetes

test-infrad – Test infrastructure for the Kubernetes project

kube-scored – Kubernetes object analysis with recommendations for improved reliability and security

Litmusd – Cloud-Native Chaos Engineering; Kubernetes-Native Chaos Engineering; Chaos Engineering for Kubernetes

PowerfulSeald – A powerful testing tool for Kubernetes clusters

kube-burnerd – Kube-burner is a tool aimed at stressing kubernetes clusters

Service Mesh

Istio – Connect, secure, control, and observe services

Traefik – The Cloud Native Edge Router

NGINX Ingress Controller – NGINX and NGINX Plus Ingress Controllers for Kubernetes

Autopilot – THE SERVICE MESH SDK

linkerd-config – A Kubernetes controller that knows how to reconcile the Linkerd configuration

Kong – Kong for Kubernetes: the official Ingress Controller for Kubernetes

OSM – Open Service Mesh (OSM) is a lightweight, extensible, cloud native service mesh

Layer5 – Layer5, the service mesh company, representing every service mesh

Gloo Mesh – The Service Mesh Orchestration Platform

APISIX – Apache APISIX is a dynamic, real-time, high-performance API gateway.

Contour – High performance ingress controller for Kubernetes

Kusk Gateway – OpenAPI-driven Ingress Controller for Kubernetes

Observability

Kubespy – Tools for observing Kubernetes resources in real time

Popeye – A Kubernetes cluster resource sanitizer

Stern – Multi pod and container log tailing for Kubernetes

Cri-tools – CLI and validation tools for Kubelet Container Runtime Interface (CRI)

Kubebox – Terminal and Web console for Kubernetes

Kubewatch – Watch k8s events and trigger Handlers

kube-state-metrics – Add-on agent to generate and expose cluster-level metrics

Sloop – Kubernetes History Visualization

kubectl tree 🎄 – Kubectl plugin to observe object hierarchies through ownerReferences

chaoskube – chaoskube periodically kills random pods in your Kubernetes cluster

BotKube – Helps you monitor your Kubernetes cluster(s), debug critical deployments and gives recommendations for standard practices

Kubestone – Kubestone is a benchmarking Operator that can evaluate the performance of Kubernetes installations

Chaos Mesh – A Chaos Engineering Platform for Kubernetes

Lemur – LEMUR: Observability and Context

kubernetes-event-exporter – Export Kubernetes events to multiple destinations with routing and filtering

Kubevious – Kubevious is an app-centric assurance, validation, and introspection platform for Kubernetes

OpenTelemetry – High-quality, ubiquitous, and portable telemetry to enable effective observability

Grafana Tempo – Grafana Tempo is a high volume, minimal dependency distributed tracing backend

Machine Learning/Deep Learning

Kubeflow – Machine Learning Toolkit for Kubernetes

Volcano – A Kubernetes Native Batch System

Compute Edge Tools

KubeEdge – Kubernetes Native Edge Computing Framework

Kubeless – Kubernetes Native Serverless Framework

Kubernetes Tools for Specific Cloud

Kubernetes on AWS (kube-aws) – A command-line tool to declaratively manage Kubernetes clusters on AWS

Draft: Streamlined Kubernetes Development – A tool for developers to create cloud-native applications on Kubernetes

helm-ssm – A low dependency tool for retrieving and injecting secrets from AWS SSM into Helm

Skupper – Multicloud communication for Kubernetes

Storage Providers

ChubaoFS – distributed file system and object storage

Longhorn – Cloud-Native distributed block storage built on and for Kubernetes

OpenEBS – Kubernetes native – hyperconverged block storage with multiple storage engines

Rook – Storage Orchestration for Kubernetes

SeaweedFS – Distributed file system supports read-write many volumes

TiKV – Distributed transactional key-value database

TopoLVM – Capacity-aware CSI plugin for Kubernetes

velero – Backup and migrate Kubernetes applications and their persistent volumes

Vitess – Vitess is a database clustering system for horizontal scaling of MySQL

kaDalu – A lightweight Persistent storage solution for Kubernetes / OpenShift using GlusterFS in background

Multiple Tools Repo

Chaos Toolkit Kubernetes Support – Kubernetes driver extension of the Chaos Toolkit probes and actions API

k14s – Kubernetes Tools that follow Unix philosophy to be simple and composable

Pulumi – Pulumi – Modern Infrastructure as Code. Any cloud, any language. Give your team cloud superpowers rocket https://www.pulumi.com

Non-Categorize

Rudr – A Kubernetes implementation of the Open Application Model specification

Keel – Kubernetes Operator to automate Helm, DaemonSet, StatefulSet & Deployment updates

Cabin, the mobile app for Kubernetes – The Mobile Dashboard for Kubernetes

Funktion – CLI tool for working with funktion

Alterant – A simple Kubernetes configuration modifier

BUCK – Brigade Universal Controller for Kubernetes

kube-fledged – A kubernetes add-on for creating and managing a cache of container images directly on the cluster worker nodes, so application pods start almost instantly

Kubecost – Cross-cloud cost allocation models for workloads running on Kubernetes

kpt – toolkit to help you manage, manipulate, customize, and apply Kubernetes Resource configuration

capsule – Capsule helps to implement a multi-tenancy and policy-based environment in your Kubernetes cluster

KubeSlice – KubeSlice enables Kubernetes pods and services to communicate seamlessly across clusters, clouds, edges, and data centers by creating logical application boundaries known as Slices

Kubernetes/K8S笔记

张琼杰阅读(741)

一、相关文档

1.Kubernetes API

https://kubernetes.io/docs/concepts/overview/kubernetes-api/

https://github.com/kubernetes/community/blob/master/contributors/devel/sig-architecture/api-conventions.md

https://github.com/kubernetes/community/blob/master/contributors/devel/sig-architecture/api_changes.md

2.Kubernetes backup and restore

https://kubernetes.io/docs/tasks/administer-cluster/configure-upgrade-etcd/#backing-up-an-etcd-cluster

https://github.com/etcd-io/website/blob/main/content/en/docs/v3.5/op-guide/recovery.md

https://www.youtube.com/watch?v=qRPNuT080Hk

3.K8S DNS

https://coredns.io/plugins/kubernetes/

https://github.com/kubernetes/dns/blob/master/docs/specification.md

4.K8S Networking

https://kubernetes.io/docs/concepts/cluster-administration/addons/

https://kubernetes.io/docs/concepts/cluster-administration/networking/#how-to-implement-the-kubernetes-networking-model

5.Kubernetes tools

Kubetools – A Curated List of Kubernetes Tools

二、相关使用

1.查看API权限

示例:

kubectl auth can-i create deployments --namespace dev
kubectl auth can-i create deployments --namespace prod
kubectl auth can-i list secrets --namespace dev --as dave

查看一个serviceAccount是否有list权限

kubectl auth can-i list pods \
	--namespace target \
	--as system:serviceaccount:dev:dev-sa

2.常用命令

1.查看所有API

_list=($(kubectl get --raw / |grep "^    \"/api"|sed 's/[",]//g')); for _api in ${_list[@]}; do _aruyo=$(kubectl get --raw ${_api} | jq .resources); if [ "x${_aruyo}" != "xnull" ]; then echo; echo "===${_api}==="; kubectl get --raw ${_api} | jq -r ".resources[].name"; fi; done

脚本

#!/bin/bash
SERVER="localhost:8080"
APIS=$(curl -s $SERVER/apis | jq -r '[.groups | .[].name] | join(" ")')
# do core resources first, which are at a separate api location
api="core"
curl -s $SERVER/api/v1 | jq -r --arg api "$api" '.resources | .[] | "\($api) \(.name): \(.verbs | join(" "))"'
# now do non-core resources
for api in $APIS; do
    version=$(curl -s $SERVER/apis/$api | jq -r '.preferredVersion.version')
    curl -s $SERVER/apis/$api/$version | jq -r --arg api "$api" '.resources | .[]? | "\($api) \(.name): \(.verbs | join(" "))"'
done

2.base64加密

echo -n "gcpuat" | base64 -w 0

3.base64解密secret内容

kubectl -n ada-datanaut get secret manual-file-upload-api-secret -o go-template='
{{range $k,$v := .data}}{{printf "%s: " $k}}{{if not $v}}{{$v}}{{else}}{{$v | base64decode}}{{end}}{{"\n"}}{{end}}'

4.密钥输出为一行

awk 'NF {sub(/\r/, ""); printf "%s\\n",$0;}' ada.chaos.mod.adag.dev.nonprod.c1.abc.com.cer

5.数据库带证书登录

mysql -u${DB_USER} -p${DB_PASSWD} -h${MYSQL_HOST} -P${MYSQL_PORT}  --ssl-ca=/tmp/certs/server-ca.pem  --ssl-cert=/tmp/certs/client-cert.pem  --ssl-key=/tmp/certs/client-key.pem

6.History配置

export HISTORY_FILE="/tmp/log/cmdlines"
export HISTFILESIZE=100000
export HISTSIZE=100000
export HISTTIMEFORMAT="$(whoami) %m/%d/%Y %H:%M "

Kubernetes/K8S基础使用方法总结【二十七】——Istio通过cert-manager配置Let'sEncrypt证书

张琼杰阅读(776)

说明:

如下实例介绍的是在istio做流量管理集群中配置Let’sEncrypte域名证书的内容。如果想要基于k8s的ingress配置Let’sEncrypte域名证书,请参考如下文章:

Kubernetes/K8S基础使用方法总结【二十三】——cert-manager的部署和使用

1.先创建issuer资源(我这里直接创建cluster-issuer来使用)

apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
  name: letsencrypt-prod-cluster
  namespace: istio-system
spec:
  acme:
    email: jakeli@jakeli.com 
    server: https://acme-v02.api.letsencrypt.org/directory 
    privateKeySecretRef:
      name: letsencrypt-prod-cluster
    solvers:
    - http01:
        ingress:
          class: istio

2.创建certificates

apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
  name: devops-jakeli
  namespace: istio-system
spec:
  secretName: devops-jakeli-cert-prod 
  duration: 2160h # 90d
  renewBefore: 360h # 15d
  isCA: false
  privateKey:
    algorithm: RSA
    encoding: PKCS1
    size: 2048
  usages:
    - server auth
    - client auth
  dnsNames:
    - "code.devops.jakeli.com"
    - "coder.devops.jakeli.com"
    - "nginx.devops.jakeli.com"
  issuerRef:
    name: letsencrypt-prod-cluster
    kind: ClusterIssuer
    group: cert-manager.io

3.创建gateway

apiVersion: networking.istio.io/v1beta1
kind: Gateway
metadata:
  labels:
    release: istio
  name: gateway 
  namespace: default 
spec:
  selector:
    app: istio-ingressgateway
    istio: ingressgateway
  servers:
  - hosts:
    - "*.devops.jakeli.com"
    port:
      name: http
      number: 80
      protocol: HTTP
  - hosts:
    - '*'
    port:
      name: https
      number: 443
      protocol: HTTPS
    tls:
      mode: SIMPLE
      credentialName: devops-jakeli-cert-prod

4.创建virtual Service

---
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: nginx 
  namespace: kube-public
spec:
  hosts:
  - "nginx.devops.jakeli.com"
  gateways:
  - "default/gateway"
  http:
  - match:
    - uri:
        exact: /
    route:
    - destination:
        host: "nginx.kube-public.svc.cluster.local"
        port:
          number: 80

参考:https://medium.com/@rd.petrusek/kubernetes-istio-cert-manager-and-lets-encrypt-c3e0822a3aaf

Cluster-API的使用方法总结

张琼杰阅读(937)

一、简述

Cluster API是kubernetes的另一个开源项目,其主要作用是基于不同的云平台或虚拟化平台而创建的CRD资源,以定义kubernetes对象资源的方式来定义、使用和管理各个平台资源的一种新型方式。下面就个人使用情况做一个记录,这里基于AWS云平台。

二、准备工作

在使用cluster api之前需要做一些准备工作,详细安装过程这里先跳过。

1.准备一个kubernetes集群,作为manager cluster。

创建配置kubernetes集群的详细步骤此处先跳过。

2.安装必要的工具及相关配置:

1)kubectl

2)docker

3)clusterctl 目前下载最新版本

curl -L https://github.com/kubernetes-sigs/cluster-api/releases/download/v1.1.3/clusterctl-linux-amd64 -o clusterctl

4)clusterawsadm

下载地址:https://github.com/kubernetes-sigs/cluster-api-provider-aws/releases

此工具主要用来解决创建aws资源的IAM权限问题,使用clusterawsadm之前需要持有administrator权限,并配置如下环境变量:

  • AWS_REGION
  • AWS_ACCESS_KEY_ID
  • AWS_SECRET_ACCESS_KEY
  • AWS_SESSION_TOKEN (如果你使用的是多因子认证需要配置)

a.然后执行如下命令来创建相关的IAM资源:

clusterawsadm bootstrap iam create-cloudformation-stack

提示:其他额外的权限配置可参考 这里

b.将上面AWS环境变量信息存储至kubernetes secret中

export AWS_B64ENCODED_CREDENTIALS=$(clusterawsadm bootstrap credentials encode-as-profile) 

5)配置默认配置文件

默认配置文件位于:$HOME/.cluster-api/clusterctl.yaml,其中可以配置provider的众多变量,如下:

[root@ip-172-31-13-197 src]# clusterctl generate provider --infrastructure aws --describe
Name: aws
Type: InfrastructureProvider
URL: https://github.com/kubernetes-sigs/cluster-api-provider-aws/releases/
Version: v1.4.0
File: infrastructure-components.yaml
TargetNamespace: capa-system
Variables:
- AUTO_CONTROLLER_IDENTITY_CREATOR
- AWS_B64ENCODED_CREDENTIALS
- AWS_CONTROLLER_IAM_ROLE
- CAPA_EKS
- CAPA_EKS_ADD_ROLES
- CAPA_EKS_IAM
- CAPA_LOGLEVEL
- EVENT_BRIDGE_INSTANCE_STATE
- EXP_BOOTSTRAP_FORMAT_IGNITION
- EXP_EKS_FARGATE
- EXP_MACHINE_POOL
- K8S_CP_LABEL
Images:
- k8s.gcr.io/cluster-api-aws/cluster-api-aws-controller:v1.4.0

若想覆盖此配置,还可以配置Overrides Layer

注意:

当同时也设置了相同名字的环境变量,环境变量具有更高的优先级。

三、初始化Manager Cluster

clusterctl初始化默认安装provider的最新可用版本(这里以AWS为例)。

[root@ip-172-31-13-197 customer]# clusterctl init --infrastructure aws --target-namespace capa
Fetching providers
Installing cert-manager Version="v1.5.3"
Waiting for cert-manager to be available...
Installing Provider="cluster-api" Version="v1.1.3" TargetNamespace="capa"
Installing Provider="bootstrap-kubeadm" Version="v1.1.3" TargetNamespace="capa"
Installing Provider="control-plane-kubeadm" Version="v1.1.3" TargetNamespace="capa"
I0417 09:20:04.840223 15472 request.go:665] Waited for 1.026955854s due to client-side throttling, not priority and fairness, request: GET:https://8AD3E49178C37D17AAE79D9114DD0D5F.gr7.us-east-1.eks.amazonaws.com/apis/controlplane.cluster.x-k8s.io/v1beta1?timeout=30s
Installing Provider="infrastructure-aws" Version="v1.4.0" TargetNamespace="capa"
Your management cluster has been initialized successfully!

You can now create your first workload cluster by running the following:

clusterctl generate cluster [name] --kubernetes-version [version] | kubectl apply -f -

如果报错如下:

[root@ip-172-31-13-197 customer]# clusterctl init --infrastructure aws --target-namespace capa
Fetching providers
Installing cert-manager Version="v1.5.3"
Error: failed to read "cert-manager.yaml" from provider's repository "cert-manager": failed to get GitHub release v1.5.3: rate limit for github api has been reached. Please wait one hour or get a personal API token and assign it to the GITHUB_TOKEN environment variable 

点击 这里 可以获取GITHUB_TOKEN的值,然后通过配置环境变量GITHUB_TOKEN来解决此报错:

# export GITHUB_TOKEN=ghp_SHNvEyOYMHw040eMlPMOYLWxLtRFsC0J

四、创建workload cluster

1.通过clusterctl工具生成workload cluster清单文件

clusterctl generate cluster capa01 \
–kubernetes-version v1.21.1 \
–control-plane-machine-count=1 \
–worker-machine-count=1 \
–flavor machinepool \
–target-namespace mycluster \
> capa01.yaml

2.创建名称空间mycluster

kubectl create ns mycluster

3.创建workload cluster

kubectl apply -f capa01.yaml

4.查看control plane状态

此时control plane还没有就绪,如下:

# kubectl get kubeadmcontrolplane -A 
NAMESPACE NAME CLUSTER INITIALIZED API SERVER AVAILABLE REPLICAS READY UPDATED UNAVAILABLE AGE VERSION
mycluster capa01-control-plane capa01 true 1 1 0 28m v1.21.1

5.获取workload capa01的kubeconfig信息到capa01.kubeconfig文件中

# clusterctl get kubeconfig capa01 > capa01.kubeconfig

6.通过kubeconfig文件为workload集群安装网络插件

这里默认是calica网络插件

# kubectl --kubeconfig=./capa01.kubeconfig   apply -f https://docs.projectcalico.org/v3.21/manifests/calico.yaml                                     
configmap/calico-config created
customresourcedefinition.apiextensions.k8s.io/bgpconfigurations.crd.projectcalico.org created
customresourcedefinition.apiextensions.k8s.io/bgppeers.crd.projectcalico.org created
customresourcedefinition.apiextensions.k8s.io/blockaffinities.crd.projectcalico.org created
customresourcedefinition.apiextensions.k8s.io/caliconodestatuses.crd.projectcalico.org created
customresourcedefinition.apiextensions.k8s.io/clusterinformations.crd.projectcalico.org created
customresourcedefinition.apiextensions.k8s.io/felixconfigurations.crd.projectcalico.org created
customresourcedefinition.apiextensions.k8s.io/globalnetworkpolicies.crd.projectcalico.org created
customresourcedefinition.apiextensions.k8s.io/globalnetworksets.crd.projectcalico.org created
customresourcedefinition.apiextensions.k8s.io/hostendpoints.crd.projectcalico.org created
customresourcedefinition.apiextensions.k8s.io/ipamblocks.crd.projectcalico.org created
customresourcedefinition.apiextensions.k8s.io/ipamconfigs.crd.projectcalico.org created
customresourcedefinition.apiextensions.k8s.io/ipamhandles.crd.projectcalico.org created
customresourcedefinition.apiextensions.k8s.io/ippools.crd.projectcalico.org created
customresourcedefinition.apiextensions.k8s.io/ipreservations.crd.projectcalico.org created
customresourcedefinition.apiextensions.k8s.io/kubecontrollersconfigurations.crd.projectcalico.org created
customresourcedefinition.apiextensions.k8s.io/networkpolicies.crd.projectcalico.org created
customresourcedefinition.apiextensions.k8s.io/networksets.crd.projectcalico.org created
clusterrole.rbac.authorization.k8s.io/calico-kube-controllers created
clusterrolebinding.rbac.authorization.k8s.io/calico-kube-controllers created
clusterrole.rbac.authorization.k8s.io/calico-node created
clusterrolebinding.rbac.authorization.k8s.io/calico-node created
daemonset.apps/calico-node created
serviceaccount/calico-node created
deployment.apps/calico-kube-controllers created
serviceaccount/calico-kube-controllers created
Warning: policy/v1beta1 PodDisruptionBudget is deprecated in v1.21+, unavailable in v1.25+; use policy/v1 PodDisruptionBudget
poddisruptionbudget.policy/calico-kube-controllers created

7.再次查看control plane的状态

如下所示,已显示为ready

# kubectl get kubeadmcontrolplane -A 
NAMESPACE NAME CLUSTER INITIALIZED API SERVER AVAILABLE REPLICAS READY UPDATED UNAVAILABLE AGE VERSION
mycluster capa01-control-plane capa01 true true 1 1 1 0 28m v1.21.1

此时可以查看集群信息,如node信息:

# kubectl --kubeconfig=./capa01.kubeconfig get nodes
NAME STATUS ROLES AGE VERSION
ip-10-0-213-197.ec2.internal Ready control-plane,master 20m v1.21.1

查看pod信息

# kubectl --kubeconfig=./capa01.kubeconfig get pods -A
NAMESPACE NAME READY STATUS RESTARTS AGE
kube-system calico-kube-controllers-fd5d6b66f-w7wck 1/1 Running 0 2m29s
kube-system calico-node-gx246 1/1 Running 0 2m29s
kube-system coredns-558bd4d5db-66jz8 1/1 Running 0 21m
kube-system coredns-558bd4d5db-vxs82 1/1 Running 0 21m
kube-system etcd-ip-10-0-213-197.ec2.internal 1/1 Running 0 20m
kube-system kube-apiserver-ip-10-0-213-197.ec2.internal 1/1 Running 0 20m
kube-system kube-controller-manager-ip-10-0-213-197.ec2.internal 1/1 Running 0 20m
kube-system kube-proxy-4fd6b 1/1 Running 0 21m
kube-system kube-scheduler-ip-10-0-213-197.ec2.internal 1/1 Running 0 20m

五、清理资源

1.删除workload cluster

kubectl delete cluster capa01

2.删除manage cluster

clusterctl delete cluster

以yaml格式导出kubernetes集群所有资源信息

张琼杰阅读(1848)

在使用kubernetes的过程中,如果有手动导出所有集群资源信息的需求,可以通过以下脚本按namespace分类自动导出。

导出后所有文件保存至tar.gz类型压缩文件中,解压后显示如下,每个目录即namespace名字,目录中的文件即此命名空间下的所有资源,以yaml格式文件存在。

完整脚本内容如下:

您暂时无权查看此隐藏内容!

Kubernetes/K8S基础使用方法总结【二十六】——变量的定义和使用

张琼杰阅读(2056)

一、简介

在编写kubernetes的manifest清单文件的时候,一般会涉及到变量的使用,用以替换yaml文件中某字段的实际值。我们在实际使用过程中,为了使目录结构简单明了,通常会结合kustomize来渲染清单文件,从而对kubernetes的对象进行控制和管理。下面就针对kustomization来对变量的定义和使用做演示。

二、文件格式变量

1.定义变量

1.在自定义文件中定义变量

在文件中以key=value的形式定义变量,多个变量可以写多行。如下,编辑名为params.env的文件,内容如下:

COP_DUMP_URL=__COP_DUMP_URL__

2.定义变量可使用位置

如下所示,列出来的项目不一定全面,如果所定义的变量无法引用可以基于如下内容进行补充。

varReference:
- path: metadata/name
  kind: Deployment
- path: metadata/namespace
  kind: Deployment
- path: metadata/labels
  kind: Deployment
- path: spec/template/metadata/labels
  kind: Deployment
- path: spec/template/spec/containers/name
  kind: Deployment
- path: spec/template/spec/containers/env/value
  kind: Deployment
- path: spec/template/spec/containers/env/valueFrom/secretKeyRef
  kind: Deployment
- path: spec/template/spec/containers/volumeMounts/name
  kind: Deployment
- path: spec/template/spec/containers/volumeMounts/mountPath
  kind: Deployment
- path: spec/template/spec/containers/envFrom/configMapRef
  kind: Deployment
- path: spec/template/spec/containers/envFrom/secretRef
  kind: Deployment
- path: spec/template/spec/volumes/configMap/name
  kind: Deployment
- path: spec/template/spec/volumes/secret/secretName
  kind: Deployment
- path: spec/template/spec/volumes/secret/items/key
  kind: Deployment
- path: spec/template/spec/volumes/secret/items/path
  kind: Deployment
- path: spec/template/spec/volumes/name
  kind: Deployment
- path: spec/selector/matchLabels
  kind: Deployment
- path: metadata/labels
  kind: Service
- path: metadata/name
  kind: Service
- path: metadata/namespace
  kind: Service
- path: metadata/annotations
  kind: Service
- path: spec/ports/name
  kind: Service
- path: spec/selector
  kind: Service
- path: metadata/name
  kind: Ingress
- path: metadata/namespace
  kind: Ingress
- path: spec/rules/http/paths/backend
  kind: Ingress
- path: spec/rules/host
  kind: Ingress
- path: spec/tls/secretName
  kind: Ingress
- path: spec/tls/hosts
  kind: Ingress
- path: metadata/name
  kind: BackendConfig
- path: metadata/namespace
  kind: BackendConfig
- path: metadata/name
  kind: Namespace
- path: metadata/name
  kind: Secret
- path: metadata/namespace
  kind: Secret
- path: data
  kind: Secret
- path: metadata/name
  kind: ConfigMap
- path: metadata/namespace
  kind: ConfigMap
- path: data
  kind: ConfigMap
- path: metadata/name
  kind: VirtualService
- path: metadata/namespace
  kind: VirtualService
- path: spec/gateways
  kind: VirtualService
- path: spec/http/route/destination/host
  kind: VirtualService
- path: metadata/name
  kind: Gateway
- path: metadata/namespace
  kind: Gateway

3.在kustomization.yaml中定义变量

这里定义的主要作用是对前面定义的parames.env和params.yaml的引用,原理是通过将变量内容创建为configmap,然后从configmap中读取变量。参考内容如下:

apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- job.yaml

vars:
- name: COP_DUMP_URL
  objref:
    kind: ConfigMap
    name: update-esa-policy-cop
    apiVersion: v1
  fieldref:
    fieldpath: data.COP_DUMP_URL
    
generatorOptions:
  disableNameSuffixHash: true
configMapGenerator:
- name: update-esa-policy-cop
  env: params.env
configurations:
- params.yaml

2.使用变量

这里使用变量就比较简单了,只需要在yaml清单文件中通过$()的方式进行引用。如下$(COP_DUMP_URL)

---
apiVersion: batch/v1
kind: Job
metadata:
  name: update-esa-policy-cop
  namespace: edsf-dsg
  labels:
    app.kubernetes.io/name: update-esa-policy-cop
    app.kubernetes.io/instance: update-esa-policy-cop
spec:
  backoffLimit: 0
  template:
    metadata:
      labels:
        app.kubernetes.io/name: update-esa-policy-cop
    spec:
      automountServiceAccountToken: false
      restartPolicy: Never
      containers:
        - args:
            - "-c"
            - "curl -k $(COP_DUMP_URL) -o /var/data/policy/cop_dump.tgz"
          command:
            - "/bin/sh" 
          name: update-esa-policy-cop
          image: update-esa-policy-cop
          imagePullPolicy: IfNotPresent
          securityContext:
            allowPrivilegeEscalation: false
            capabilities:
              drop:
              - ALL
            privileged: false
          volumeMounts:
            - name: policy-storage
              mountPath: /var/data/policy
              subPath: policy-storage
          resources:
            limits:
              cpu: 500m
              memory: 3500Mi
            requests:
              cpu: 200m
              memory: 256Mi
      volumes:
        - name: policy-storage
          persistentVolumeClaim:
            claimName: dsg-policy-pv-claim

三、pod信息变量

将pod信息作为变量传递给容器,一般有两种使用场景:

1.用 Pod 字段作为环境变量的值

参考代码如下:

apiVersion: v1
kind: Pod
metadata:
name: dapi-envars-fieldref
spec:
containers:
- name: test-container
image: k8s.gcr.io/busybox
command: [ "sh", "-c"]
args:
- while true; do
echo -en '\n';
printenv MY_NODE_NAME MY_POD_NAME MY_POD_NAMESPACE;
printenv MY_POD_IP MY_POD_SERVICE_ACCOUNT;
sleep 10;
done;
env:
- name: MY_NODE_NAME
valueFrom:
fieldRef:
fieldPath: spec.nodeName
- name: MY_POD_NAME
valueFrom:
fieldRef:
fieldPath: metadata.name
- name: MY_POD_NAMESPACE
valueFrom:
fieldRef:
fieldPath: metadata.namespace
- name: MY_POD_IP
valueFrom:
fieldRef:
fieldPath: status.podIP
- name: MY_POD_SERVICE_ACCOUNT
valueFrom:
fieldRef:
fieldPath: spec.serviceAccountName
restartPolicy: Never

这个配置文件中,你可以看到五个环境变量。env 字段是一个 EnvVars. 对象的数组。 数组中第一个元素指定 MY_NODE_NAME 这个环境变量从 Pod 的 spec.nodeName 字段获取变量值。 同样,其它环境变量也是从 Pod 的字段获取它们的变量值。

2.用 Container 字段作为环境变量的值

参考代码如下:

apiVersion: v1
kind: Pod
metadata:
name: dapi-envars-resourcefieldref
spec:
containers:
- name: test-container
image: k8s.gcr.io/busybox:1.24
command: [ "sh", "-c"]
args:
- while true; do
echo -en '\n';
printenv MY_CPU_REQUEST MY_CPU_LIMIT;
printenv MY_MEM_REQUEST MY_MEM_LIMIT;
sleep 10;
done;
resources:
requests:
memory: "32Mi"
cpu: "125m"
limits:
memory: "64Mi"
cpu: "250m"
env:
- name: MY_CPU_REQUEST
valueFrom:
resourceFieldRef:
containerName: test-container
resource: requests.cpu
- name: MY_CPU_LIMIT
valueFrom:
resourceFieldRef:
containerName: test-container
resource: limits.cpu
- name: MY_MEM_REQUEST
valueFrom:
resourceFieldRef:
containerName: test-container
resource: requests.memory
- name: MY_MEM_LIMIT
valueFrom:
resourceFieldRef:
containerName: test-container
resource: limits.memory
restartPolicy: Never

这个配置文件中,你可以看到四个环境变量。env 字段是一个 EnvVars. 对象的数组。数组中第一个元素指定 MY_CPU_REQUEST 这个环境变量从 Container 的 requests.cpu 字段获取变量值。同样,其它环境变量也是从 Container 的字段获取它们的变量值。

Kubernetes/K8S基础使用方法总结【二十五】——垃圾回收

张琼杰阅读(899)

参考kubernetes官方文档:Garbage Collection | Kubernetes

1.获取所有kubernetes资源

kubectl api-resources --verbs=list -o name | xargs -n 1 kubectl get --all-namespaces -o=json | jq -c '.items[] | {name: .metadata.name, kind: .kind, ownerReferences:  .metadata.ownerReferences }'

2.获取所有kubernetes资源的metadata.ownerReferenceskubernetes

kubectl api-resources --verbs=list -o name | xargs -n 1 kubectl get --all-namespaces -o=json | jq -c '.items[] | {name: .metadata.name, kind: .kind, ownerReferences: select( has ("ownerReferences") ).ownerReferences }'

3.清理方式

如何清理kubernetes集群内部的垃圾信息呢?这里以清理无用的pv为例进行说明。思路:通过创建kubernetes对象cronjob来定期进行清理,完整yaml清单文件如下:

您暂时无权查看此隐藏内容!

 

分享交流,合作共赢!

联系我们加入QQ群