Heteki no space error

8/8/2018

I'm trying to setup a glusterfs cluster with kubernetes. I managed to start the glusterd pods on all the nodes (3 nodes) I also managed to load the topology successfully, however when I run

heketi-cli setup-openshift-heketi-storage

I get the following error:

Error: No space

This is the output of

heketi-cli topology load --json=gluster-kubernetes/deploy/topology.json

        Found node vps01 on cluster 1a36667e4275773fc353f2caaaaaaa                                                                                       
                Adding device /dev/loop0 ... OK                                                                                                             
        Found node vps02 on cluster 1a36667e4275773fc353faaaaaaaa                                                                                       
                Found device /dev/loop0                                                                                                                     
        Found node vps04 on cluster 1a36667e4275773fc353faaaaaaa                                                                                      
                Adding device /dev/loop0 ... OK       

Output of

heketi-cli topology info 

Cluster Id: 1a36667e4275773fc353f2caaaaaa                                                                                                         

    File:  true                                                                                                                                             
    Block: true                                                                                                                                             

    Volumes:                                                                                                                                                

    Nodes:                                                                                                                                                  

        Node Id: 1752dcf447c8eb6eaad45aaaa                                                                                                         
        State: online                                                                                                                                       
        Cluster Id: 1a36667e4275773fc353f2caaa                                                                                                    
        Zone: 1                                                                                                                                             
        Management Hostnames: vps01                                                                                                                         
        Storage Hostnames: XX.XX.XX.219                                                                                                                    
        Devices:                                                                                                                                            
                Id:50396d72293c4723504810108bd75d41   Name:/dev/loop0          State:online    Size (GiB):12      Used (GiB):0       Free (GiB):12          
                        Bricks:                                                                                                                             

        Node Id: 56b8c1942b347a863ee73a005758cc27                                                                                                           
        State: online                                                                                                                                       
        Cluster Id: 1a36667e4275773fc353f2c8eb2dd2a3                                                                                                        
        Zone: 1                                                                                                                                             
        Management Hostnames: vps04                                                                                                                         
        Storage Hostnames: XX.XX.XX.227                                                                                                                     
        Devices:                                                                                                                                            
                Id:dc75ad8154234ebcf9174b018d0bc30a   Name:/dev/loop0          State:online    Size (GiB):9       Used (GiB):4       Free (GiB):5           
                        Bricks:                                                                                                                             

        Node Id: f82cb81a026884764d3d953c7c9b6a9f                                                                                                           
        State: online                                                                                                                                       
        Cluster Id: 1a36667e4275773fc353f2c8eb2dd2a3                                                                                                        
        Zone: 1                                                                                                                                             
        Management Hostnames: vps02                                                                                                                         
        Storage Hostnames: XX.XX.XX.157                                                                                                                     
        Devices:                                                                                                                                            
                Id:1914102b7ae395f12797981a0e3cf5a4   Name:/dev/loop0          State:online    Size (GiB):4       Used (GiB):4       Free (GiB):0           
                        Bricks:   

There is no more space on device 1914102b7ae395f12797981a0e3cf5a4, however I didn't not store anything yet on the device.

For info here is the topology.json file:

{
  "clusters": [
    {
      "nodes": [
        {
          "node": {
            "hostnames": {
              "manage": [
                "vps01"
              ],
              "storage": [
                "XX.XX.XX.219"
              ]
            },
            "zone": 1
          },
          "devices": [
            "/dev/loop0"
          ]
        },
        {
          "node": {
            "hostnames": {
              "manage": [
                "vps02"
              ],
              "storage": [
                "XX.XX.XX.157"
              ]
            },
            "zone": 1                                                                                                                                       
          },                                                                                                                                                
          "devices": [                                                                                                                                      
            "/dev/loop0"                                                                                                                                    
          ]                                                                                                                                                 
        },                                                                                                                                                  
        {                                                                                                                                                   
          "node": {                                                                                                                                         
            "hostnames": {                                                                                                                                  
              "manage": [                                                                                                                                   
                "vps04"                                                                                                                                     
              ],                                                                                                                                            
              "storage": [                                                                                                                                  
                "XX.XX.XX.227"                                                                                                                              
              ]                                                                                                                                             
            },                                                                                                                                              
            "zone": 1                                                                                                                                       
          },                                                                                                                                                
          "devices": [                                                                                                                                      
            "/dev/loop0"                                                                                                                                    
          ]                                                                                                                                                 
        }                                                                                                                                                   
      ]                                                                                                                                                     
    }                                                                                                                                                       
  ]                                                                                                                                                         
}   
-- Ben D
glusterfs
kubernetes

1 Answer

8/8/2018

You can try this:

# ./gk-deploy -g --abort 
# dmsetup remove_all   # In each server.
# dmsetup ls
# rm -fr /var/lib/glusterd/vols/* # In each server.
# rm -fr /var/lib/heketi/* # In each server.
# wipefs -a /dev/<device> # In each server.

source: https://github.com/gluster/gluster-kubernetes/issues/369#issuecomment-383247722

-- kumar
Source: StackOverflow