I can’t find information about where the pods that run on nodes are stored… I know its temporary file but what if I want them to be created on specific storage or mount point ?
From df -h on one node I can see it mounts:
shm 64M 0 64M 0% /run/k3s/containerd/io.containerd.grpc.v1.cri/sandboxes/a9d11117b319432fff833cda48e5dc16f88bcc28b8e8148077d607ddaf41dfab/shm
overlay 15G 1.9G 12G 14% /run/k3s/containerd/io.containerd.runtime.v1.linux/k8s.io/a9d11117b319432fff833cda48e5dc16f88bcc28b8e8148077d607ddaf41dfab/rootfs
overlay 15G 1.9G 12G 14% /run/k3s/containerd/io.containerd.runtime.v1.linux/k8s.io/ab8b17764bed69e2702565fa171fcc84defcab1de7093bd68f53581732ef39fb/rootfs
overlay 15G 1.9G 12G 14% /run/k3s/containerd/io.containerd.runtime.v1.linux/k8s.io/dc246f55de330a4a0d23da56bf7212008e49fe974302c664b930869544a58051/rootfs
tmpfs 396M 0 396M 0% /run/user/1000But where is the setting what root folder it use for that mount… what I want to do is specify for example /cluster_tmp to be the main root for the cluster pods that are deployed on node…
I’m running rpi cluster and would like to use usb3 disk for the storage not the sd card, to lighten the read/writes to the card…
It's not like K3s will create pods in specific directory on node. I would recommend you to read about Kubernetes architecture and Kubernetes components.
If you want to use specific directory on the node or some network drive you have to read Persistent Volume and Persistent Volume Claim.
Here you can find information how specify which path in container will indicate path in your VM/network disk.
Please elaborate your question if this answer is not what you are looking for.
/run should be tmpfs, thus be in ram instead of being on the physical media. The directories reported there are not the actual storage.
Given your goal of storing most things on the usb storage device, the simplest solution would be to move the entire k3s data directory to it, using the --data-dir/-d command line option as per the Documentation. This will force k3s, and the embedded containerd, to place all of their working files and directory on the usb storage.
I believe k3s comes with its own containerd binary and related configuration. By installing k3s explicitly, that is by invoking the "k3s server" command, with the option
--container-runtime-endpoint value (agent/runtime) Disable embedded containerd and use alternative CRI implementationyou could specify an alternate container runtime that is configured according to your needs, For example with storage in a different volume or partition.
More info in the k3s documentation
Regarding your specific idea though, what happens if the usb stick disappears?
k3s stands on the shoulders of giants. As container-runtime it uses containerd. And containerd gets configured with its own configuration toml. See man 5 containerd-config:
root : The root directory for containerd metadata. (Default: "/var/lib/containerd")And this is how /etc/containerd/config.toml looks like:
root = "/var/lib/containerd"
state = "/run/containerd"
oom_score = 0
imports = ["/etc/containerd/runtime_*.toml", "./debug.toml"]
[grpc]
...
The root parameter is the most interesting to you. Here you can find its description:
root will be used to store any type of persistent data for containerd. Snapshots, content, metadata for containers and image, as well as any plugin data will be kept in this location. The root is also namespaced for plugins that containerd loads. Each plugin will have its own directory where it stores data. containerd itself does not actually have any persistent data that it needs to store, its functionality comes from the plugins that are loaded.
/var/lib/containerd/
├── io.containerd.content.v1.content
│ ├── blobs
│ └── ingest
├── io.containerd.metadata.v1.bolt
│ └── meta.db
├── io.containerd.runtime.v1.linux
│ ├── default
│ └── example
├── io.containerd.snapshotter.v1.btrfs
└── io.containerd.snapshotter.v1.overlayfs
├── metadata.db
└── snapshots