rook-ceph fio benchmark with ioengine=rbd

11/10/2021

I have deployed storageclass.yaml found under rook/cluster/examples/kubernetes/ceph/csi/rbd/ directory and created a PVC claim. I need to fio benchmark with ioengine=rbd. In my fio config file I need to set the following:

clientname= 
pool=
rbdname=

I looked in the storageclass.yaml for what the appropriate values might be. My best guess is as follows.

clientname=admin
pool=replicapool
rbdname=rook-ceph-block

Does this look correct ?

-- user3304297
ceph
kubernetes
librbd
rook-storage

1 Answer

12/16/2021

When you use rook-ceph to provision storage for your k8s cluster, you don't use ceph interfaces directly via ceph clients (eg. librbd), but rook will run these clients for you and make the storage available for your container so that the containerized app doesn't have to care about ceph.

So for example if your goal is to run fio benchmark on a block device provisioned by rook ceph (so it will be backed by ceph rbd under the hood), you need to make sure you have rook-ceph-block storage class created, and then you can:

Ask for such raw block PV via PVC which would look like this:

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: block-storage
spec:
  accessModes:
  - ReadWriteOnce
  resources:
    requests:
      storage: 4Gi
  storageClassName: rook-ceph-block
  volumeMode: Block

And then reference this pvc/block-storage in a container spec like this (it's a rough example, but hopefully you get the idea):

    spec:
      containers:
      - command:
        - /usr/bin/fio
        - --blocksize=4k
        - --ioengine=libaio
        - --filename=/dev/target
        image: fio:latest
        name: fio
        volumeDevices:
        - devicePath: /dev/target
          name: block-storage
      restartPolicy: Never
      volumes:
      - name: block-storage
        persistentVolumeClaim:
          claimName: block-storage
-- marbu
Source: StackOverflow