Chris Hall bio photo

Chris Hall

Principal Technical Consultant

PolarCloudsUK Chris LinkedIn Github
Chris Hall Nutanix Certified Master - Multicloud Infrastructure 6 Chris Hall VMware vExpert 2024 Chris Hall VMware vExpert NSX 2023 Chris Hall Nutanix Certified Professional - Multicloud Infrastructure 6 Chris Hall Nutanix Certified Professional - Unified Storage 6 Chris Hall VMware vExpert 2023 Chris Hall VMware vExpert 2022

Failed Volume File this one away under “I hope I won’t ever need this, but just in case here it is anyway”.

As a last resort before investigating data restoration from backup(s), or even starting to look for data recovery services, the vSphere On-disk Metadata Analyzer (VOMA) tool can be used to identify and fix incidents of logical volume metadata corruption.

In this post we will take a closer look at VOMA.

Overview

Symptoms

The following may be observed:

  • No access to files located on a datastore
  • Reports of metadata errors in the vmkernel.log file

Risks

VOMA is potentially a tool of last resort and as so it should be treated as such. Because VOMA requires all VMs on a suspect volume to either have been powered off or migrated away from the suspect volume (if possible), there are situations where data may become permanently unavailable. Because of this it is always advisable to have current backups of VMs in an environment.

There may be situations where the deletion and recreation of a datastore may be preferable over a repair attempt.

Using VOMA

OK, so you understand the risks, you’ve migrated and/or powered down your VMs and you are ready. Lets look at using the tool.

First open an SSH connection to your ESXi host and issue the command:

voma

VOMA Usage

To find the device name of the volume that is having the issues, use the command:

esxcli storage vmfs extent list

esxcli storage vmfs extent list

As you can see for the above screenshot, my datastore DATASTORE-01 has a device name of mpx.vmhba0:C0:T1:L0.

As VOMA must be run against the partition rather then the device, we need to suffix our device name with :1. Therefore my device becomes mpx.vmhba0:C0:T1:L0:1.

First off we will check for errors on my VMFS volume using the following command:

 voma -m vmfs -f check -d /vmfs/devices/disks/mpx.vmhba0:C0:T1:L0:1

VOMA Check Mode

In my test environment screenshot above, I’m not seeing any errors reported by VOMA. If you are reading this post, you will probably be seeing a different result.

If you are and you don’t have current good backups and/or you don’t already have a ticket open with VMware support, then I advise you STOP HERE and REVIEW.

To continue to try and fix errors with VOMA (usual disclaimers apply!), the following command may be used:

 voma -m vmfs -f fix -d /vmfs/devices/disks/mpx.vmhba0:C0:T1:L0:1

VOMA Fix Mode

Further Reading

The following resources offer further information on using VOMA:

Conclusion and Wrap Up

In this post we took a closer look at the vSphere On-disk Metadata Analyzer (VOMA) tool.

Will VOMA help me in my situation?

Well only you can make that call. If you are already looking at data restoration / recovery situation, then VOMA might just be able to help.

-Chris