Once in a while and for various reasons, a vSAN object will become inaccessible. Problem is: The inaccessible objects occupies space on the vSAN Datastore, but is not counted in overall capacity use as seen by Skyline health and Capacity. I recently dealt with just such an issue where Skyline Health and Capacity were showing 32% available vSAN capacity, but in Datastores the actually availability was just a few hundred GB, less than 5% of the vSAN capacity.
As it turns out, this was because of a large inaccessible object (also determined to be not needed/obsolete) which was visible in vSAN Virtual Objects.
Here’s the solution:
- Identify the UID of the inaccessible object in vCenter, under Virtual Objects
- Log in to any ESXi Host in the cluster with SSH
- Run the following command to identify the friendly name and properties of the inaccessible object:
- /usr/lib/vmware/osfs/bin/objtool getAttr -u <OBJECT UID>
- If that fails to retrieve any info, run this command against the UID of the inaccessible object:
- /usr/lib/vmware/osfs/bin/objtool getAttr -u <OBJECT UID> –bypassDom
- Verify the friendly name that the inaccessible object belonged to and, if appropriate, proceed to remove that object
- /usr/lib/vmware/osfs/bin/objtool delete -u <OBJECT UID> -f
Here’s what it looks like in practice:
1 | [root@esxi011:~] /usr/lib/vmware/osfs/bin/objtool getAttr -u f47e275f-8cc3-f865-c458-00a38e8e2fea |
2 | Failed to get object attributes : Input/output error 327684. |
3 | object getAttr error: Failure |
4 | [root@esxi011:~] /usr/lib/vmware/osfs/bin/objtool getAttr -u f47e275f-8cc3-f865-c458-00a38e8e2fea --bypassDom |
7 | UUID:f47e275f-8cc3-f865-c458-00a38e8e2fea |
11 | Object size:32212254720000 |
13 | User friendly name:(null) |
19 | Policy:((\"stripeWidth\" i1) (\"cacheReservation\" i0) (\"proportionalCapacity\" i0) (\"hostFailuresToTolerate\" i1) (\"forceProvisioning\" i0) (\"spbmProfileId\" \"1d04bc75-ffe7-4316-adbf-ddc8f2168bb7\") (\"spbmProfileGenerationNumber\" l+0) (\"replicaPreference\" \"Capacity\") (\"iopsLimit\" i0) (\"checksumDisabled\" i0) (\"spbmProfileName\" \"Backup Volume\")) |
23 | Object capabilities: NONE |
25 | Object path: /vmfs/volumes/vsan:526186362509cc06-d35c848bf26e6e22/71363c5e-6c76-e8a7-df90-00be75494ac8/old_veeam_repo_1_1.vmdk |
27 | Group uuid: 71363c5e-6c76-e8a7-df90-00be75494ac8 |
31 | [root@esxi011:~] /usr/lib/vmware/osfs/bin/objtool delete -u f47e275f-8cc3-f865-c458-00a38e8e2fea -f |