VMware Workstation - How to compress/cleanup VMDK?

[besweeet]

My 10.8.3 VM ended up using the entire 150GB VMDK space, even though the OS itself uses less than 10GB. Since VMware Workstation can't compress/reclaim space/cleanup HFS+ VMDKs, how would one get the space back without having to start over? I've tried adding an empty VMDK to the VM and cloning it using a number of different methods (Disk Utility, Carbon Copy Cloner, SuperDuper!, Terminal), yet none of them managed to get the new VMDK to boot properly.

 

Any ideas?

[Zenith432]

You need to zero unused sectors before compressing.  When compressing a vmdk, it eliminates sectors only if they're all zero.  Otherwise it can't know if the sector is occupied or not.

So go to any folder on the volume in the guest and run

cat /dev/zero >zero.fill; sleep 1; sync; sleep 1; /bin/rm -f zero.fill

Then shutdown the guest and run VMware's compression on the vmdk.

 

Edit: note that it is not perfect, because there's some discarded space inside HFS metadata - like free inodes and such that are no longer used - but will not be zeroed with this.  Not sure whether there's a tool for HFS to free unused metadata space.

[besweeet]

Thanks, but after a while, Terminal said this:

cat: stdout: No space left on device

 

Not sure how to continue.

[Zenith432]

quit terminal, shutdown the virtual machine, and run the VMware compact on the vmdk.

Not sure how to continue.

 

[besweeet]

quit terminal, shutdown the virtual machine, and run the VMware compact on the vmdk.

Oops, forgot to post this earlier: I ended up doing exactly that, and it worked! Went from nearly 150GB for the VMDK down to less than 9GB.

[MSoK]

I use Disk Utility to "Erase Free Space", select your virtual hard disk in DU select the Erase tab and click on the "Erase Free Space..." button I leave the default setting of "Fastest" and click on the "Erase Free Space" button and wait for the Erasing Free space to complete, it will take some time. Once complete shutdown the OS X VM, and edit Virtual Machine Settings, select the Hard Disk and select "Compact" from the "Utilities" pull down menu, a progress bar will appear to indicate the "Compacting virtual disk..." has started, again this will take some time, but at the end of the process the hard drive VMDK should have shrunk in size.

[besweeet]

I use Disk Utility to "Erase Free Space", select your virtual hard disk in DU select the Erase tab and click on the "Erase Free Space..." button I leave the default setting of "Fastest" and click on the "Erase Free Space" button and wait for the Erasing Free space to complete, it will take some time. Once complete shutdown the OS X VM, and edit Virtual Machine Settings, select the Hard Disk and select "Compact" from the "Utilities" pull down menu, a progress bar will appear to indicate the "Compacting virtual disk..." has started, again this will take some time, but at the end of the process the hard drive VMDK should have shrunk in size.

I remember using Erase Free Space when the VMDK was at around 55GB, and it ended up filling the entire thing... I probably had the wrong setting. I'll try that the next time it fills itself up, thanks.

[besweeet]

*massive bump*

 

Using WS11 and Yosemite, that command seems to do what it usually does, but after clicking on WS11's compact button for the VMDK, it instantly says that it's been finished while it doesn't actually do anything. Any ideas?

[Zenith432]

I have a worse problem which is that when I try to compact from vmware GUI, I get a popup with an error message "A required file was not found". I've searched the net for the popup, and it's mentioned but I couldn't find a solution.

 

What I do is compact the vmdk from the command line. Open a cmd window, and go to the directory where the vmware binaries are. Then run

vmware-vdiskmanager -k <full-path-to-vmdk>

When I use the command line version, it also prints a warning about a missing file, but then proceeds to compact the vmdk.

 

Go figure...

 

Using WS11 and Yosemite, that command seems to do what it usually does, but after clicking on WS11's compact button for the VMDK, it instantly says that it's been finished while it doesn't actually do anything. Any ideas?

[besweeet]

I have a worse problem which is that when I try to compact from vmware GUI, I get a popup with an error message "A required file was not found". I've searched the net for the popup, and it's mentioned but I couldn't find a solution.

 

What I do is compact the vmdk from the command line. Open a cmd window, and go to the directory where the vmware binaries are. Then run

vmware-vdiskmanager -k <full-path-to-vmdk>

When I use the command line version, it also prints a warning about a missing file, but then proceeds to compact the vmdk.

 

Go figure...

 

That command worked for me, thanks.

[Donk]

*massive bump*

 

Using WS11 and Yosemite, that command seems to do what it usually does, but after clicking on WS11's compact button for the VMDK, it instantly says that it's been finished while it doesn't actually do anything. Any ideas?

 

Compact only seems to work for filing systems such as NTFS and FAT32. Does not work with Linux VMs.

[Zenith432]

You need to wipe the free space on the target filesystem to zeros.  For Linux, can use a bash command like in post #2 above.  It works for me with ext4.  I don't know of Linux tools to wipe free space other than the bash method.

 

The vmware compaction eliminates all-zero sectors from the vmdk sparse files.  That's why they need to be wiped first.

On Windows there's a tool called sdelete that can be used with -z switch to wipe free space to zero.  It's part of sysinternals suite.

 

There are also commercial tools like Active KillDisk, Jetico BCWipe that do a better job of wiping filesystem metadata.  The bash script only wipes free allocation blocks, but not metadata.  In OS X there's the "Erase Free Space" which MSoK mentioned above (post #6) - I think it also wipes only free allocation blocks, not metadata.

 

Compact only seems to work for filing systems such as NTFS and FAT32. Does not work with Linux VMs.

[Donk]

You need to wipe the free space on the target filesystem to zeros.  For Linux, can use a bash command like in post #2 above.  It works for me with ext4.  I don't know of Linux tools to wipe free space other than the bash method.

 

The vmware compaction eliminates all-zero sectors from the vmdk sparse files.  That's why they need to be wiped first.

On Windows there's a tool called sdelete that can be used with -z switch to wipe free space to zero.  It's part of sysinternals suite.

 

There are also commercial tools like Active KillDisk, Jetico BCWipe that do a better job of wiping filesystem metadata.  The bash script only wipes free allocation blocks, but not metadata.  In OS X there's the "Erase Free Space" which MSoK mentioned above (post #6) - I think it also wipes only free allocation blocks, not metadata.

 

Sorry I was mistakenly talking about something different. I actually was referring to the "Clean Up Disks" menu item, which I confused with the  compact options. You can still use VMware tools from the command line to do much of this.

 

Windows: VMwareToolboxCmd.exe

Linux: vmare-toolbox-cmd

OSX: vmware-tools-cli

 

Run the command with "help disk" to see the options.

[Zenith432]

I don't see any "Clean Up Disks".  I see "defragment" and "compact".

 

Not sure what "defragment" does.  I use the contig tool from sysinternals, and after "defragment" it reports the vmdk still fragmented as a file.  "defragment" refers to some vmdk internal fragmentation - and I don't know what's the benefit of it.  After doing "compact" - doing "defragment" says it's already defragmented.

 

Thanks for the tip about vmware-toolbox-cmd.  I'll try it.

[Donk]

I don't see any "Clean Up Disks".  I see "defragment" and "compact".

 

 

It's on the VM-->Manage menu in Workstation 10 and 11. Also similar functionality in Fusion on first tab of the Settings dialog.

[Zenith432]

Ok, I never noticed that "Clean Up Disks"   Have no idea what it does.  Just tells me that "no cleanup is necessary".

 

I tried wiping ext4 using vmware-toolbox-cmd, and then compacting - works, with result comparable to what I usually get with the bash command from post #2. 

[besweeet]

Another bump...

 

I tried using vmware-vdiskmanager -k, but got the following:

VixDiskLib: Invalid configuration file parameter. Failed to read configuration file.
Failed to shrink the disk '[redacted].vmdk' : An error occurred while writing a file; the disk is full. Data has not been saved. Free some disk space and try again (0x8).

None of my physical drivers are full, nor is the VMDK. Unless it needs enough free space to match the existing VMDK size?

[justvisiting]

I had enough local disk space, so I created another blank VM disk in vmware player, attached it to the  OSX virtual machine and used Carbon Copy Cloner to make an identical bootable copy. 

 

Once complete, I disconnected the original disk and verified the cloned disk booted successfully and that I could my regular tasks.

 

then I deleted the old vm disk.   

 

very fast on SSDs  

[besweeet]

I had enough local disk space, so I created another blank VM disk in vmware player, attached it to the  OSX virtual machine and used Carbon Copy Cloner to make an identical bootable copy. 

 

Once complete, I disconnected the original disk and verified the cloned disk booted successfully and that I could my regular tasks.

 

then I deleted the old vm disk.   

 

very fast on SSDs  

That would be a decent alternative -- thanks.

[RDPetruska]

Another bump...

None of my physical drivers are full, nor is the VMDK. Unless it needs enough free space to match the existing VMDK size?

 

Yes, that's one of the main benefits of using the split disk vmdk files.  Compress, cleanup, cloneiing, defragment operations on the vmdk file requires (internally) making a working copy of the file being operated on... if it's a 2GB split disk, then you only need slightly more than 2 GB of free space, as VMware works on each slice independently.  If it's a monolithic (single file) disk, you need at least as much free space as the *maximum defined file size* of the VMDK - even if it isn't using that much!

[besweeet]

Yes, that's one of the main benefits of using the split disk vmdk files.  Compress, cleanup, cloneiing, defragment operations on the vmdk file requires (internally) making a working copy of the file being operated on... if it's a 2GB split disk, then you only need slightly more than 2 GB of free space, as VMware works on each slice independently.  If it's a monolithic (single file) disk, you need at least as much free space as the *maximum defined file size* of the VMDK - even if it isn't using that much!

In that case, it must not be cleaning properly because the drive the VMDK is on (an SSD) doesn't have enough free space to properly complete? I'll transfer it to my HDD sometime and try it out.

[LuXs]

try this http://www.insanelymac.com/forum/topic/308367-how-to-shrink-os-x-vmdk/see if that helps you out.
Tested it on mavericks, yosemite and el capitan

[besweeet]

try this http://www.insanelymac.com/forum/topic/308367-how-to-shrink-os-x-vmdk/see if that helps you out.

Tested it on mavericks, yosemite and el capitan

I've already been replying to that thread .

[besweeet]

You were right -- it did have to do with not having enough free space on the VMDK's drive. I moved my VMDK to a larger drive, ran the command, and it shrunk the VMDK just fine. From 39.3GB to 33.7GB.

 

c:\Program Files (x86)\VMware\VMware Workstation>vmware-vdiskmanager.exe -k "s:\OS X 10.10 Yosemite.vmdk"
VixDiskLib: Invalid configuration file parameter. Failed to read configuration file.
  Shrink: 100% done.
Shrink completed successfully.