Différences

Ci-dessous, les différences entre deux révisions de la page.

--- tech:notes_k8s_-_kind_pour_awx [2025/12/12 09:24] – Jean-Baptiste
+++ tech:notes_k8s_-_kind_pour_awx [2026/03/03 13:51] (Version actuelle) – Jean-Baptiste
@@ Ligne 1: / Ligne 1: @@
+<!DOCTYPE markdown>
+{{tag>Brouillon K8S Ansible K8S}}
+# Notes k8s - kind pour AWX
+Voir :
+* [[Pb cgroup controllers not delegated to non-privileged users - SystemD delegate failed - cgroupControllers]]
+* https://kind.sigs.k8s.io/docs/user/rootless/
+* https://docs.ansible.com/projects/awx-operator/en/latest/installation/kind-install.html
+* https://une-tasse-de.cafe/blog/kind/
+* https://ansible.readthedocs.io/projects/awx-operator/en/latest/user-guide/advanced-configuration/custom-volume-and-volume-mount-options.html
+* https://wiki.evolix.org/HowtoAWX
+Voir aussi :
+  * kinder
+  * https://github.com/kurokobo/awx-on-k3s
+Avantage / Intérêts d'utiliser AWX :
+* https://ansible.readthedocs.io/projects/awx/en/24.6.1/userguide/overview.html
+* http://docs.ansible.com/automation-controller/latest/html/controllercli/usage.html - CI/CD
+* Gestion des secrets (natif ou hachicorp Vault)
+* Traçabilité logs
+* Scalabilité
+* RBAC
+* API REST
+* Empêcher les accès concurrentiels
+* WorkFlow AWX
+* Pouvoir rejouer le même playbook, avec exactement les mêmes entrés en garantissant la reproductibilité via un Execution Environment
+* Ansible Galaxy Integration (et gestion dépendances des collections / rôles)
+* Contrôle des extravars (surveys)
+* Planification (schedules)
+Reducing complexity
+* EE : software dependencies, portability, content separation
+Source : https://blog.stephane-robert.info/post/ansible-awx-operator-installation-kubernetes/
+Voir aussi :
+* [[Notes Kubernetes k8s - Install]]
+* https://blog.stephane-robert.info/post/ansible-awx-getting-started/
+* https://kubernetes-team.pages.debian.net/docs/packages/kind.html
+* https://www.spinkube.dev/blog/2024/04/01/spinkube-on-kind-rancher-desktop-super-powers/
+How to reference a local volume in Kind (kubernetes in container)
+* https://stackoverflow.com/questions/62694361/how-to-reference-a-local-volume-in-kind-kubernetes-in-docker
+Il faut containerd  >= 1.7
+Une fois le cluster crée il n'est pas possible de le modifier.
+Prévoir de pouvoir restaurer la conf près un "delete" et un "create"
+Voir aussi :
+* containers-storage
+Kube kind
+https://github.com/containerd/nerdctl
+https://kind.sigs.k8s.io/docs/user/rootless/
+~~~bash
+KIND_EXPERIMENTAL_PROVIDER=nerdctl
+# nerdctl / kind a besoin de connaitre le chemin de iptables
+export PATH=$PATH:/usr/sbin/
+~~~
+Install de kind via asdf (voir aussi mise.run)
+~~~bash
+asdf plugin add kind
+asdf install kind latest
+asdf set --home kind latest
+~~~
+Pour Podman
+Si nouvelle partition
+~~~bash
+mkfs.xfs -n ftype=1 -m reflink=1 /dev/mapper/vg_data-data
+~~~
+Pour le rootless : pas de homedir monté en noexec/nodev
+source : https://github.com/containers/podman/blob/main/rootless.md
+~~~bash
+export KIND_EXPERIMENTAL_PROVIDER=podman
+systemd-run --scope --user kind create cluster
+~~~
+Rootless
+Enable cgroup v2 delegation, see https://rootlesscontaine.rs/getting-started/common/cgroup2/ . This step is required; the rootless kubelet will fail to start without the proper cgroups delegated.
+Source : https://docs.k3s.io/advanced
+Pour rootless avec iptables
+`/etc/modules-load.d/iptables.conf`
+~~~
+ip6_tables
+ip6table_nat
+ip_tables
+iptable_nat
+~~~
+Si nftable à la place d'iptables
+~~~yaml
+kind: Cluster
+apiVersion: kind.x-k8s.io/v1alpha4
+networking:
+  ipFamily: ipv4
+  kubeProxyMode: "nftables"
+~~~
+Pour nerdctl
+~~~
+KIND_EXPERIMENTAL_PROVIDER=nerdctl kind create cluster
+~~~
+A titre d'exemple.
+Mais idéalement il faudrait ajouter un ''extraPortMappings'' et ''ExtraMount''
+~~~
+Running as unit: run-p8566-i8567.scope; invocation ID: 066b7253045348e79515baad90cd38ad
+Creating cluster "kind" ...
+ ✓ Ensuring node image (kindest/node:v1.34.0) 🖼
+ ✓ Preparing nodes 📦
+ ✓ Writing configuration 📜
+ ✓ Starting control-plane 🕹
+ ✓ Installing CNI 🔌
+ ✓ Installing StorageClass 💾
+Set kubectl context to "kind-kind"
+You can now use your cluster with:
+kubectl cluster-info --context kind-kind
+Not sure what to do next? 😅  Check out https://kind.sigs.k8s.io/docs/user/quick-start/
+~~~
+install Nginx Ingress controller
+~~~bash
+kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/main/deploy/static/provider/kind/deploy.yaml
+~~~
+------
+Get kind images list :
+~~~bash
+nerdctl exec -ti kind-control-plane crictl images
+~~~
+Cet logs
+~~~bash
+kind export logs
+~~~
+------
+~~~bash
+kubectl port-forward svc/awx-service 3000:80
+~~~
+https://stackoverflow.com/questions/62432961/how-to-use-nodeport-with-kind
+------
+Delete
+https://stackoverflow.com/questions/55672498/kubernetes-cluster-stuck-on-removing-pv-pvc
+Erreur pv pvc
+https://www.datree.io/resources/kubernetes-troubleshooting-fixing-persistentvolumeclaims-error
+https://www.kubernet.dev/fixing-pod-has-unbound-immediate-persistentvolumeclaims-error-in-kubernetes-a-detailed-guide/
+### Démarage auto
+Ne fonctionne pas
+~~~bash
+nerdctl update --restart unless-stopped kind-control-plane
+~~~
+Serait-ce lié à https://github.com/containerd/nerdctl/issues/2286
+Voir aussi ''podman generate systemd --new --files --name kind-control-plane'' et **Quadlets** https://www.redhat.com/en/blog/quadlet-podman
+''~/.config/systemd/user/container-kind-control-plane.service''
+~~~ini
+[Unit]
+Description=Container kind-control-plane
+Wants=network-online.target
+After=network-online.target
+#RequiresMountsFor=%t/containers
+[Service]
+Delegate=yes
+#Type=notify
+Type=oneshot
+RemainAfterExit=yes
+#RemainAfterExit=yes
+Environment=PODMAN_SYSTEMD_UNIT=%n
+Environment=DBUS_SESSION_BUS_ADDRESS=unix:path=/run/user/1003/bus
+Environment=XDG_RUNTIME_DIR=/run/user/1003
+ExecStartPre=/bin/bash -c '/usr/bin/podman stop kind-control-plane &'
+ExecStartPre=/usr/bin/sleep 5
+#Restart=on-failure
+RemainAfterExit=yes
+#TimeoutStopSec=60
+ExecStart=/bin/bash -c '/usr/bin/podman start kind-control-plane &'
+ExecStop=/bin/bash -c '/usr/bin/podman stop kind-control-plane &'
+NotifyAccess=all
+[Install]
+WantedBy=default.target
+~~~
+Voir aussi :
+* https://feldspaten.org/2022/07/09/On-the-joy-of-podman-and-auto-updates/
+* https://github.com/containers/podman/discussions/20218#discussioncomment-7169693
+* https://unix.stackexchange.com/questions/714167/best-practices-for-running-a-rootless-container-as-a-systemd-service-with-user
+~~~bash
+systemctl --user daemon-reload
+systemctl --user enable container-kind-control-plane.service
+~~~
+## Pb
+### Err failed to create fsnotify watcher - too many open files
+~~~bash
+kubectl get pods
+kubectl logs -f awx-764564987d-wtw2f
+failed to create fsnotify watcher: too many open files
+~~~
+Solution
+''/etc/sysctl.d/10-k8s.conf''
+~~~python
+# Raise inotify resource limits
+fs.inotify.max_user_instances = 1024
+fs.inotify.max_user_watches = 524288
+~~~
+~~~bash
+sysctl -p /etc/sysctl.d/10-k8s.conf
+~~~
+## AWX
+Voir :
+  * https://ansible.readthedocs.io/projects/awx-operator/en/latest/installation/basic-install.html
+  * https://ansible.readthedocs.io/projects/awx-operator/en/latest/installation/kind-install.html
+~~~bash
+kubectl get secret awx-admin-password -o jsonpath="{.data.password}" | base64 --decode ; echo
+~~~
+''~/.bashrc''
+~~~bash
+function awx-manage() {
+        # podman exec -ti kind-control-plane crictl exec -ti --name awx-task awx-manage "$@"
+	nerdctl exec -ti kind-control-plane -- crictl exec -ti --name awx-task awx-manage "$@"
+}
+~~~
+~~~bash
+nerdctl exec -ti kind-control-plane -- crictl exec -ti --name awx-task /bin/bash
+cd /tmp/
+python3 -m venv ipython
+cd ipython/
+source bin/activate
+pip install ipython
+export PYTHONPATH=/tmp/ipython/lib/python3.6/site-packages/
+awx-manage shell_plus --ipython
+~~~
+------
+~~~yaml
+kind: Cluster
+apiVersion: kind.x-k8s.io/v1alpha4
+nodes:
+- role: control-plane
+  extraMounts:
+    - containerPath: /data/postgres-13
+      hostPath: /data/postgres-13
+    - containerPath: /files
+      hostPath: /data/files
+      readOnly: true
+    - containerPath: /shares
+      hostPath: /data/shares
+# Si module nf_tables est présent à la pace de iptables
+networking:
+  kubeProxyMode: "nftables"
+~~~
+~~~bash
+kind create cluster --config cluster-config.yml
+kubectl get pods -A -w
+git clone https://github.com/ansible/awx-operator.git
+cd awx-operator
+export NAMESPACE=awx
+kubectl create ns ${NAMESPACE}
+kubectl config set-context --current --namespace=$NAMESPACE
+export RELEASE_TAG=`curl -s https://api.github.com/repos/ansible/awx-operator/releases/latest | grep tag_name | cut -d '"' -f 4`
+git checkout $RELEASE_TAG
+cd config/manager
+~/code/awx-operator/bin/kustomize edit set image controller=quay.io/ansible/awx-operator:0.14.0
+cd ~/code/awx-operator/
+~/code/awx-operator/bin/kustomize build config/default | kubectl apply -f -
+kubectl get pods -w
+tee awx-pv.yml <<EOF
+---
+apiVersion: v1
+kind: PersistentVolume
+metadata:
+  name: awx-postgres-13-volume
+spec:
+  accessModes:
+    - ReadWriteOnce
+  persistentVolumeReclaimPolicy: Retain
+  capacity:
+    storage: 8Gi
+  storageClassName: local-path
+  hostPath:
+    path: /data/postgres-13
+---
+apiVersion: v1
+kind: PersistentVolume
+metadata:
+  name: awx-projects-volume
+spec:
+  accessModes:
+    - ReadWriteOnce
+  persistentVolumeReclaimPolicy: Retain
+  capacity:
+    storage: 2Gi
+  storageClassName: local-path
+  hostPath:
+    path: /data/projects
+EOF
+kubectl apply -f awx-pv.yml
+kubectl get pv -w
+tee awx-pvc.yml <<EOF
+---
+apiVersion: v1
+kind: PersistentVolumeClaim
+metadata:
+  name: awx-projects-claim
+spec:
+  accessModes:
+    - ReadWriteOnce
+  volumeMode: Filesystem
+  resources:
+    requests:
+      storage: 2Gi
+  storageClassName: local-path
+EOF
+kubectl apply -f awx-pvc.yml
+kubectl get pvc -w
+tee awx-deployment.yml<<EOF
+---
+apiVersion: awx.ansible.com/v1beta1
+kind: AWX
+metadata:
+  name: awx
+spec:
+  # These parameters are designed for use with AWX Operator 0.29.0
+  # and AWX 21.6.0
+  postgres_configuration_secret: awx-postgres-configuration
+  postgres_storage_class: local-path
+  postgres_storage_requirements:
+    requests:
+      storage: 8Gi
+  projects_persistence: true
+  garbage_collect_secrets: false
+  projects_existing_claim: awx-projects-claim
+  postgres_init_container_resource_requirements: {}
+  postgres_resource_requirements: {}
+  web_resource_requirements: {}
+  task_resource_requirements: {}
+  ee_resource_requirements: {}
+  service_type: ClusterIP
+  ingress_type: ingress
+  hostname: awx.robert.local
+  ingress_annotations: |
+    kubernetes.io/ingress.class: traefik
+EOF
+kubectl apply -f awx-deployment.yml
+kubectl get AWX -w
+# nerdctl exec -ti kind-control-plane bash
+# mkdir /data/postgres-13
+~~~