Space Reclamation In-Guest

The discussion above speaks only about space reclamation directly on a VMFS volume which pertains to dead space accumulated by the deletion or migration of virtual machines and virtual disks. Running UNMAP on a VMFS only removes dead space in that scenario. But, as mentioned earlier, dead space can accumulate higher up in the VMware stack—inside of the virtual machine itself.

1. The target virtual disk must be a thin virtual disk. Thick-type virtual disks do not support UNMAP.
2. For Windows In-Guest UNMAP:
   1. ESXi 6.0 and later
   2. VM Hardware version 11 and later
3. For Linux In-Guest UNMAP:
   1. ESXi 6.5 and later
   2. VM Hardware version 13 and later
4. If Change Block Tracking (CBT) is enabled for a virtual disk, In-Guest UNMAP for that virtual disk is only supported starting with ESXi 6.5

   In-Guest UNMAP Alignment Requirements

   VMware ESXi requires that any UNMAP request sent down by a guest must be aligned to 1 MB. For a variety of reasons, not all UNMAP requests will be aligned as such and in in ESXi 6.5 and earlier a large percentage failed. In ESXi 6.5 Patch 1, ESXi has been altered to be more tolerant of misaligned UNMAP requests. See the VMware patch information here.

   Prior to this, any UNMAP requests that were even partially misaligned would fail entirely. Leading to no reclamation. In ESXi 6.5 P1, any portion of UNMAP requests that are aligned will be accepted and passed along to the underlying array. Misaligned portions will be accepted but not passed down. Instead, the affected blocks referred to by the misaligned UNMAPs will be instead zeroed out with WRITE SAME. The benefit of this behavior on the FlashArray, is that zeroing is identical in behavior to UNMAP so all of the space will be reclaimed regardless of misalignment.

   Note:

   BEST PRACTICE: Apply ESXi 6.5 Patch Release ESXi650-201703001 (2148989) as soon as possible to be able to take full advantage of in-guest UNMAP.

   In-Guest UNMAP with Windows

   Starting with ESXi 6.0, In-Guest UNMAP is supported with Windows 2012 R2 and later Windows-based operating systems. For a full report of UNMAP support with Windows, please refer to Microsoft documentation.

   NTFS supports automatic UNMAP by default—this means (assuming the underlying storage supports it) Windows will issue UNMAP to the blocks a file used to consume immediately once it has been deleted or moved.

   Automatic UNMAP is enabled by default in Windows. This can be verified with the following CLI command:

   
   fsutil behavior query DisableDeleteNotify

   If DisableDeleteNotify is set to 0, UNMAP is ENABLED. Setting it to 1, DISABLES it. Everpure recommends this value remain enabled. To change it, use the following command:

   
   fsutil behavior set DisableDeleteNotify 0

   

   Windows also supports manual UNMAP, which can be run on-demand or per a schedule. This is performed using the Disk Optimizer tool. Thin virtual disks can be identified in the tool as volume media types of “thin provisioned drive”—these are the volumes that support UNMAP.

   

   Select the drive and click “Optimize”. Or configure a scheduled optimization.

   Windows prior to ESXi 6.5 Patch 1

   Ordinarily, this would work with the default configuration of NTFS, but VMware enforces additional UNMAP alignment, that requires a non-default NTFS configuration. In order to enable in-guest UNMAP in Windows for a given NTFS, that NTFS must be formatted using a 32 or 64K allocation unit size. This will force far more Windows UNMAP operations to be aligned with VMware requirements.

   

   64K is also the standard recommendation for SQL Server installations—which therefore makes this a generally accepted change. To checking existing NTFS volumes are using the proper allocation unit size to support UNMAP, this simple PowerShell two-line command can be run to list a report:

   
   $wql = "SELECT Label, Blocksize, Name FROM Win32_Volume WHERE FileSystem='NTFS'"
   Get-WmiObject -Query $wql -ComputerName '.' | Select-Object Label, Blocksize, Name
   

   

   Note:

   BEST PRACTICE: Use the 32 or 64K Allocation Unit Size for NTFS to enable automatic UNMAP in a Windows virtual machine.

   Due to alignment issues, the manual UNMAP tool (Disk Optimizer) is not particularly effective as often most UNMAPs are misaligned and will fail.

   Windows with ESXi 6.5 Patch 1 and Later

   As of ESXi 6.5 Patch 1, all NTFS allocation unit sizes will work with in-guest UNMAP. So at this ESXi level no unit size change is required to enable this functionality. That being said, there is additional benefit to using a 32 or 64 K allocation unit. While all sizes will allow all space to be reclaimed on the FlashArray, a 32 or 64 K allocation unit will cause more UNMAP requests to be aligned and therefore more of the underlying virtual disk will be returned to the VMFS (more of it will be shrunk).

   The manual tool, Disk Optimizer, now works quite well and can be used. If UNMAP is disabled in Windows (it is enabled by default) this tool can be used to reclaim space on-demand or via a schedule. If automatic UNMAP is enabled, there is generally no need to use this tool.

   For more information on this, please read the following blog post here.

   In-Guest UNMAP with Linux

   Starting with ESXi 6.5, In-Guest UNMAP is supported with Linux-based operating systems and most common file systems (Ext4, Btrfs, JFS, XFS, F2FS, VFAT). For a full report of UNMAP support with Linux configurations, please refer to appropriate Linux distribution documentation. To enable this behavior it is necessary to use Virtual Machine Hardware Version 13 or later.

   Linux supports both automatic and manual methods of UNMAP.

   Linux file systems do not support automatic UNMAP by default—this behavior needs to be enabled during the mount operation of the file system. This is achieved by mounting the file system with the “discard” option.

   
   pureuser@ubuntu:/mnt$ sudo mount /dev/sdd /mnt/unmaptest -o discard

   When mounted with the discard option, Linux will issue UNMAP to the blocks a file used to consume immediately once it has been deleted or moved.

   Everpure does not require this feature to be enabled, but generally recommends doing so to keep capacity information correct throughout the storage stack.

   Note:

   BEST PRACTICE: Mount Linux filesystems with the “discard” option to enable in-guest UNMAP for Linux-based virtual machines.

   Linux with ESXi 6.5

   In ESXi 6.5, automatic UNMAP is supported and is able to reclaim most of the identified dead space. In general, Linux aligns most UNMAP requests in automatic UNMAP and therefore is quite effective in reclaiming space.

   The manual method fstrim, does align initial UNMAP requests and therefore entirely fails.

   

   Linux with ESXi 6.5 Patch 1 and Later

   In ESXi 6.5 Patch 1 and later, automatic UNMAP is even more effective, now that even the small number of misaligned UNMAPs are handled. Furthermore, the manual method via fstrim works as well. So in this ESXi version, either method is a valid option.