Notes k8s - kind pour AWX

Voir :

https://kind.sigs.k8s.io/docs/user/rootless/

https://ansible.readthedocs.io/projects/awx-operator/en/latest/installation/kind-install.html#kind-install

https://une-tasse-de.cafe/blog/kind/

https://ansible.readthedocs.io/projects/awx-operator/en/latest/user-guide/advanced-configuration/custom-volume-and-volume-mount-options.html

Voir aussi :

kinder

Avantage / Intérêts d'utiliser AWX :

https://ansible.readthedocs.io/projects/awx/en/24.6.1/userguide/overview.html

http://docs.ansible.com/automation-controller/latest/html/controllercli/usage.html - CI/CD

Gestion des secrets (natif ou hachicorp Vault)

Traçabilité logs

Scalabilité

RBAC

API REST

Empêcher les accès concurrentiels

WorkFlow AWX

Pouvoir rejouer le même playbook, avec exactement les mêmes entrés en garantissant la reproductibilité via un Execution Environment

Ansible Galaxy Integration (et gestion dépendances des collections / rôles)

Contrôle des extravars (surveys)

Planification (schedules)

Source : https://blog.stephane-robert.info/post/ansible-awx-operator-installation-kubernetes/

Voir aussi :

Notes Kubernetes k8s - Install

https://blog.stephane-robert.info/post/ansible-awx-getting-started/

https://kubernetes-team.pages.debian.net/docs/packages/kind.html

https://www.spinkube.dev/blog/2024/04/01/spinkube-on-kind-rancher-desktop-super-powers/

How to reference a local volume in Kind (kubernetes in container)

https://stackoverflow.com/questions/62694361/how-to-reference-a-local-volume-in-kind-kubernetes-in-docker

Il faut containerd >= 1.7

Une fois le cluster crée il n'est pas possible de le modifier. Prévoir de pouvoir restaurer la conf près un “delete” et un “create”

Voir aussi :

containers-storage

Kube kind

https://github.com/containerd/nerdctl

https://kind.sigs.k8s.io/docs/user/rootless/

KIND_EXPERIMENTAL_PROVIDER=nerdctl

Install de kind via asdf

asdf plugin add kind asdf install kind latest asdf set --home kind latest

Pour Podman

Si nouvelle partition

mkfs.xfs -n ftype=1 -m reflink=1 /dev/mapper/vg_data-data

Pour le rootless : pas de homedir monté en noexec/nodev source : https://github.com/containers/podman/blob/main/rootless.md

export KIND_EXPERIMENTAL_PROVIDER=podman systemd-run --scope --user kind create cluster

Pour nerdctl

KIND_EXPERIMENTAL_PROVIDER=nerdctl kind create cluster

A titre d'exemple. Mais idéalement il faudrait ajouter un extraPortMappings et ExtraMount

Running as unit: run-p8566-i8567.scope; invocation ID: 066b7253045348e79515baad90cd38ad
Creating cluster "kind" ...
✓ Ensuring node image (kindest/node:v1.34.0) 🖼
✓ Preparing nodes 📦
✓ Writing configuration 📜
✓ Starting control-plane 🕹
✓ Installing CNI 🔌
✓ Installing StorageClass 💾
Set kubectl context to "kind-kind"
You can now use your cluster with:
kubectl cluster-info --context kind-kind
Not sure what to do next? 😅 Check out https://kind.sigs.k8s.io/docs/user/quick-start/

Get kind images list :

nerdctl exec -ti kind-control-plane crictl images

Cet logs

kind export logs

kubectl port-forward svc/awx-service 3000:80

https://stackoverflow.com/questions/62432961/how-to-use-nodeport-with-kind

Delete

https://stackoverflow.com/questions/55672498/kubernetes-cluster-stuck-on-removing-pv-pvc

Erreur pv pvc https://www.datree.io/resources/kubernetes-troubleshooting-fixing-persistentvolumeclaims-error

https://www.kubernet.dev/fixing-pod-has-unbound-immediate-persistentvolumeclaims-error-in-kubernetes-a-detailed-guide/

Démarage auto

Ne fonctionne pas

nerdctl update --restart unless-stopped kind-control-plane

Serait-ce lié à https://github.com/containerd/nerdctl/issues/2286

Voir aussi podman generate systemd --new --files --name kind-control-plane et Quadlets https://www.redhat.com/en/blog/quadlet-podman

~/.config/systemd/user/container-kind-control-plane.service

[Unit] Description=Container kind-control-plane Wants=network-online.target After=network-online.target #RequiresMountsFor=%t/containers [Service] Delegate=yes #Type=notify Type=oneshot RemainAfterExit=yes #RemainAfterExit=yes Environment=PODMAN_SYSTEMD_UNIT=%n Environment=DBUS_SESSION_BUS_ADDRESS=unix:path=/run/user/1003/bus Environment=XDG_RUNTIME_DIR=/run/user/1003 ExecStartPre=/bin/bash -c '/usr/bin/podman stop kind-control-plane &' ExecStartPre=/usr/bin/sleep 5 #Restart=on-failure RemainAfterExit=yes #TimeoutStopSec=60 ExecStart=/bin/bash -c '/usr/bin/podman start kind-control-plane &' ExecStop=/bin/bash '/usr/bin/podman stop kind-control-plane &' NotifyAccess=all [Install] WantedBy=default.target

Voir aussi :

https://feldspaten.org/2022/07/09/On-the-joy-of-podman-and-auto-updates/

https://github.com/containers/podman/discussions/20218#discussioncomment-7169693

https://unix.stackexchange.com/questions/714167/best-practices-for-running-a-rootless-container-as-a-systemd-service-with-user

systemctl --user daemon-reload systemctl --user enable container-kind-control-plane.service

Pb

Err failed to create fsnotify watcher: too many open files

kubectl get pods kubectl logs -f awx-764564987d-wtw2f failed to create fsnotify watcher: too many open files

Solution

#sudo sysctl -w fs.inotify.max_user_watches=2099999999
#sudo sysctl -w fs.inotify.max_user_instances=2099999999
#sudo sysctl -w fs.inotify.max_queued_events=2099999999
#sysctl -w fs.inotify.max_user_watches=1048576
#sysctl -w fs.inotify.max_user_instances=8192

/etc/sysctl.d/10-k8s.conf

# Raise inotify resource limits fs.inotify.max_user_instances = 1024 fs.inotify.max_user_watches = 524288
sysctl -p /etc/sysctl.d/10-k8s.conf

# AWX

Voir :

https://ansible.readthedocs.io/projects/awx-operator/en/latest/installation/basic-install.html

https://ansible.readthedocs.io/projects/awx-operator/en/latest/installation/kind-install.html

kubectl get secret awx-admin-password -o jsonpath="{.data.password}" | base64 --decode ; echo

~/.bashrc

function awx-manage() { # podman exec -ti kind-control-plane crictl exec -ti --name awx-task awx-manage "$@" nerdctl exec -ti kind-control-plane -- crictl exec -ti --name awx-task awx-manage "$@" }
nerdctl exec -ti kind-control-plane -- crictl exec -ti --name awx-task /bin/bash cd /tmp/ python3 -m venv ipython cd ipython/ source bin/activate pip install ipython export PYTHONPATH=/tmp/ipython/lib/python3.6/site-packages/ awx-manage shell_plus --ipython