{{tag>Brouillon K8S Ansible K8S}} # Notes k8s - kind pour AWX Voir : * [[Pb cgroup controllers not delegated to non-privileged users - SystemD delegate failed - cgroupControllers]] * https://kind.sigs.k8s.io/docs/user/rootless/ * https://docs.ansible.com/projects/awx-operator/en/latest/installation/kind-install.html * https://une-tasse-de.cafe/blog/kind/ * https://ansible.readthedocs.io/projects/awx-operator/en/latest/user-guide/advanced-configuration/custom-volume-and-volume-mount-options.html * https://wiki.evolix.org/HowtoAWX Voir aussi : * kinder Avantage / Intérêts d'utiliser AWX : * https://ansible.readthedocs.io/projects/awx/en/24.6.1/userguide/overview.html * http://docs.ansible.com/automation-controller/latest/html/controllercli/usage.html - CI/CD * Gestion des secrets (natif ou hachicorp Vault) * Traçabilité logs * Scalabilité * RBAC * API REST * Empêcher les accès concurrentiels * WorkFlow AWX * Pouvoir rejouer le même playbook, avec exactement les mêmes entrés en garantissant la reproductibilité via un Execution Environment * Ansible Galaxy Integration (et gestion dépendances des collections / rôles) * Contrôle des extravars (surveys) * Planification (schedules) Reducing complexity * EE : software dependencies, portability, content separation Source : https://blog.stephane-robert.info/post/ansible-awx-operator-installation-kubernetes/ Voir aussi : * [[Notes Kubernetes k8s - Install]] * https://blog.stephane-robert.info/post/ansible-awx-getting-started/ * https://kubernetes-team.pages.debian.net/docs/packages/kind.html * https://www.spinkube.dev/blog/2024/04/01/spinkube-on-kind-rancher-desktop-super-powers/ How to reference a local volume in Kind (kubernetes in container) * https://stackoverflow.com/questions/62694361/how-to-reference-a-local-volume-in-kind-kubernetes-in-docker Il faut containerd >= 1.7 Une fois le cluster crée il n'est pas possible de le modifier. Prévoir de pouvoir restaurer la conf près un "delete" et un "create" Voir aussi : * containers-storage Kube kind https://github.com/containerd/nerdctl https://kind.sigs.k8s.io/docs/user/rootless/ ~~~bash KIND_EXPERIMENTAL_PROVIDER=nerdctl # nerdctl / kind a besoin de connaitre le chemin de iptables export PATH=$PATH:/usr/sbin/ ~~~ Install de kind via asdf ~~~bash asdf plugin add kind asdf install kind latest asdf set --home kind latest ~~~ Pour Podman Si nouvelle partition ~~~bash mkfs.xfs -n ftype=1 -m reflink=1 /dev/mapper/vg_data-data ~~~ Pour le rootless : pas de homedir monté en noexec/nodev source : https://github.com/containers/podman/blob/main/rootless.md ~~~bash export KIND_EXPERIMENTAL_PROVIDER=podman systemd-run --scope --user kind create cluster ~~~ Pour rootless avec iptables `/etc/modules-load.d/iptables.conf` ~~~ ip6_tables ip6table_nat ip_tables iptable_nat ~~~ Si nftable à la place d'iptables ~~~yaml kind: Cluster apiVersion: kind.x-k8s.io/v1alpha4 networking: ipFamily: ipv4 kubeProxyMode: "nftables" ~~~ Pour nerdctl ~~~ KIND_EXPERIMENTAL_PROVIDER=nerdctl kind create cluster ~~~ A titre d'exemple. Mais idéalement il faudrait ajouter un ''extraPortMappings'' et ''ExtraMount'' ~~~ Running as unit: run-p8566-i8567.scope; invocation ID: 066b7253045348e79515baad90cd38ad Creating cluster "kind" ... ✓ Ensuring node image (kindest/node:v1.34.0) 🖼 ✓ Preparing nodes 📦 ✓ Writing configuration 📜 ✓ Starting control-plane 🕹 ✓ Installing CNI 🔌 ✓ Installing StorageClass 💾 Set kubectl context to "kind-kind" You can now use your cluster with: kubectl cluster-info --context kind-kind Not sure what to do next? 😅 Check out https://kind.sigs.k8s.io/docs/user/quick-start/ ~~~ install Nginx Ingress controller ~~~bash kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/main/deploy/static/provider/kind/deploy.yaml ~~~ ------ Get kind images list : ~~~bash nerdctl exec -ti kind-control-plane crictl images ~~~ Cet logs ~~~bash kind export logs ~~~ ------ ~~~bash kubectl port-forward svc/awx-service 3000:80 ~~~ https://stackoverflow.com/questions/62432961/how-to-use-nodeport-with-kind ------ Delete https://stackoverflow.com/questions/55672498/kubernetes-cluster-stuck-on-removing-pv-pvc Erreur pv pvc https://www.datree.io/resources/kubernetes-troubleshooting-fixing-persistentvolumeclaims-error https://www.kubernet.dev/fixing-pod-has-unbound-immediate-persistentvolumeclaims-error-in-kubernetes-a-detailed-guide/ ### Démarage auto Ne fonctionne pas ~~~bash nerdctl update --restart unless-stopped kind-control-plane ~~~ Serait-ce lié à https://github.com/containerd/nerdctl/issues/2286 Voir aussi ''podman generate systemd --new --files --name kind-control-plane'' et **Quadlets** https://www.redhat.com/en/blog/quadlet-podman ''~/.config/systemd/user/container-kind-control-plane.service'' ~~~ini [Unit] Description=Container kind-control-plane Wants=network-online.target After=network-online.target #RequiresMountsFor=%t/containers [Service] Delegate=yes #Type=notify Type=oneshot RemainAfterExit=yes #RemainAfterExit=yes Environment=PODMAN_SYSTEMD_UNIT=%n Environment=DBUS_SESSION_BUS_ADDRESS=unix:path=/run/user/1003/bus Environment=XDG_RUNTIME_DIR=/run/user/1003 ExecStartPre=/bin/bash -c '/usr/bin/podman stop kind-control-plane &' ExecStartPre=/usr/bin/sleep 5 #Restart=on-failure RemainAfterExit=yes #TimeoutStopSec=60 ExecStart=/bin/bash -c '/usr/bin/podman start kind-control-plane &' ExecStop=/bin/bash '/usr/bin/podman stop kind-control-plane &' NotifyAccess=all [Install] WantedBy=default.target ~~~ Voir aussi : * https://feldspaten.org/2022/07/09/On-the-joy-of-podman-and-auto-updates/ * https://github.com/containers/podman/discussions/20218#discussioncomment-7169693 * https://unix.stackexchange.com/questions/714167/best-practices-for-running-a-rootless-container-as-a-systemd-service-with-user ~~~bash systemctl --user daemon-reload systemctl --user enable container-kind-control-plane.service ~~~ ## Pb ### Err failed to create fsnotify watcher - too many open files ~~~bash kubectl get pods kubectl logs -f awx-764564987d-wtw2f failed to create fsnotify watcher: too many open files ~~~ Solution ''/etc/sysctl.d/10-k8s.conf'' ~~~python # Raise inotify resource limits fs.inotify.max_user_instances = 1024 fs.inotify.max_user_watches = 524288 ~~~ ~~~bash sysctl -p /etc/sysctl.d/10-k8s.conf ~~~ ## AWX Voir : * https://ansible.readthedocs.io/projects/awx-operator/en/latest/installation/basic-install.html * https://ansible.readthedocs.io/projects/awx-operator/en/latest/installation/kind-install.html ~~~bash kubectl get secret awx-admin-password -o jsonpath="{.data.password}" | base64 --decode ; echo ~~~ ''~/.bashrc'' ~~~bash function awx-manage() { # podman exec -ti kind-control-plane crictl exec -ti --name awx-task awx-manage "$@" nerdctl exec -ti kind-control-plane -- crictl exec -ti --name awx-task awx-manage "$@" } ~~~ ~~~bash nerdctl exec -ti kind-control-plane -- crictl exec -ti --name awx-task /bin/bash cd /tmp/ python3 -m venv ipython cd ipython/ source bin/activate pip install ipython export PYTHONPATH=/tmp/ipython/lib/python3.6/site-packages/ awx-manage shell_plus --ipython ~~~ ------ ~~~yaml kind: Cluster apiVersion: kind.x-k8s.io/v1alpha4 nodes: - role: control-plane extraMounts: - containerPath: /data/postgres-13 hostPath: /data/postgres-13 - containerPath: /files hostPath: /data/files readOnly: true - containerPath: /shares hostPath: /data/shares # Si module nf_tables est présent à la pace de iptables networking: kubeProxyMode: "nftables" ~~~ ~~~bash kind create cluster --config cluster-config.yml kubectl get pods -A -w git clone https://github.com/ansible/awx-operator.git cd awx-operator export NAMESPACE=awx kubectl create ns ${NAMESPACE} kubectl config set-context --current --namespace=$NAMESPACE export RELEASE_TAG=`curl -s https://api.github.com/repos/ansible/awx-operator/releases/latest | grep tag_name | cut -d '"' -f 4` git checkout $RELEASE_TAG cd config/manager ~/code/awx-operator/bin/kustomize edit set image controller=quay.io/ansible/awx-operator:0.14.0 cd ~/code/awx-operator/ ~/code/awx-operator/bin/kustomize build config/default | kubectl apply -f - kubectl get pods -w tee awx-pv.yml <