Identifying and removing inaccessible vSAN objects
Once in a while and for various reasons, a vSAN object will become inaccessible. Problem is: The inaccessible objects occupies space on the vSAN Datastore, but is not counted in overall capacity use as seen by Skyline health and Capacity. I recently dealt with just such an issue where Skyline Health and Capacity were showing 32% available vSAN capacity, but in Datastores the actually availability was just a few hundred GB, less than 5% of the vSAN capacity.
As it turns out, this was because of a large inaccessible object (also determined to be not needed/obsolete) which was visible in vSAN Virtual Objects.
Here’s the solution:
- Identify the UID of the inaccessible object in vCenter, under Virtual Objects
- Log in to any ESXi Host in the cluster with SSH
- Run the following command to identify the friendly name and properties of the inaccessible object:
- /usr/lib/vmware/osfs/bin/objtool getAttr -u <OBJECT UID>
- If that fails to retrieve any info, run this command against the UID of the inaccessible object:
- /usr/lib/vmware/osfs/bin/objtool getAttr -u <OBJECT UID> –bypassDom
- Run the following command to identify the friendly name and properties of the inaccessible object:
- Verify the friendly name that the inaccessible object belonged to and, if appropriate, proceed to remove that object
- /usr/lib/vmware/osfs/bin/objtool delete -u <OBJECT UID> -f
Here’s what it looks like in practice:
[root@esxi011:~] /usr/lib/vmware/osfs/bin/objtool getAttr -u f47e275f-8cc3-f865-c458-00a38e8e2fea Failed to get object attributes : Input/output error 327684. object getAttr error: Failure [root@esxi011:~] /usr/lib/vmware/osfs/bin/objtool getAttr -u f47e275f-8cc3-f865-c458-00a38e8e2fea --bypassDom Object Attributes -- UUID:f47e275f-8cc3-f865-c458-00a38e8e2fea Object type:vsan Object size:32212254720000 User friendly name:(null) HA metadata:(null) Allocation type:Thin Policy:((\"stripeWidth\" i1) (\"cacheReservation\" i0) (\"proportionalCapacity\" i0) (\"hostFailuresToTolerate\" i1) (\"forceProvisioning\" i0) (\"spbmProfileId\" \"1d04bc75-ffe7-4316-adbf-ddc8f2168bb7\") (\"spbmProfileGenerationNumber\" l+0) (\"replicaPreference\" \"Capacity\") (\"iopsLimit\" i0) (\"checksumDisabled\" i0) (\"spbmProfileName\" \"Backup Volume\")) Object class: vdisk Object capabilities: NONE Object path: /vmfs/volumes/vsan:526186362509cc06-d35c848bf26e6e22/71363c5e-6c76-e8a7-df90-00be75494ac8/old_veeam_repo_1_1.vmdk Group uuid: 71363c5e-6c76-e8a7-df90-00be75494ac8 Container uuid: (null) [root@esxi011:~] /usr/lib/vmware/osfs/bin/objtool delete -u f47e275f-8cc3-f865-c458-00a38e8e2fea -f [root@esxi011:~]