Jump to content

VMware Workstation - How to compress/cleanup VMDK?


  • Please log in to reply
18 replies to this topic

#1
besweeet

besweeet

    InsanelyMac Geek

  • Members
  • PipPipPip
  • 172 posts
  • Gender:Male

My 10.8.3 VM ended up using the entire 150GB VMDK space, even though the OS itself uses less than 10GB. Since VMware Workstation can't compress/reclaim space/cleanup HFS+ VMDKs, how would one get the space back without having to start over? I've tried adding an empty VMDK to the VM and cloning it using a number of different methods (Disk Utility, Carbon Copy Cloner, SuperDuper!, Terminal), yet none of them managed to get the new VMDK to boot properly.

 

Any ideas?



#2
Zenith432

Zenith432

    InsanelyMac Legend

  • Developers
  • 501 posts
  • Gender:Male

You need to zero unused sectors before compressing.  When compressing a vmdk, it eliminates sectors only if they're all zero.  Otherwise it can't know if the sector is occupied or not.

So go to any folder on the volume in the guest and run

cat /dev/zero >zero.fill; sleep 1; sync; sleep 1; /bin/rm -f zero.fill

Then shutdown the guest and run VMware's compression on the vmdk.

 

Edit: note that it is not perfect, because there's some discarded space inside HFS metadata - like free inodes and such that are no longer used - but will not be zeroed with this.  Not sure whether there's a tool for HFS to free unused metadata space.



#3
besweeet

besweeet

    InsanelyMac Geek

  • Members
  • PipPipPip
  • 172 posts
  • Gender:Male

Thanks, but after a while, Terminal said this:

cat: stdout: No space left on device
 
Not sure how to continue.


#4
Zenith432

Zenith432

    InsanelyMac Legend

  • Developers
  • 501 posts
  • Gender:Male

quit terminal, shutdown the virtual machine, and run the VMware compact on the vmdk.

Not sure how to continue.

 



#5
besweeet

besweeet

    InsanelyMac Geek

  • Members
  • PipPipPip
  • 172 posts
  • Gender:Male

quit terminal, shutdown the virtual machine, and run the VMware compact on the vmdk.

Oops, forgot to post this earlier: I ended up doing exactly that, and it worked! Went from nearly 150GB for the VMDK down to less than 9GB.



#6
MSoK

MSoK

    InsanelyMac Legend

  • Members
  • PipPipPipPipPipPipPip
  • 999 posts
  • Gender:Male
  • Location:Darlington UK

I use Disk Utility to "Erase Free Space", select your virtual hard disk in DU select the Erase tab and click on the "Erase Free Space..." button I leave the default setting of "Fastest" and click on the "Erase Free Space" button and wait for the Erasing Free space to complete, it will take some time. Once complete shutdown the OS X VM, and edit Virtual Machine Settings, select the Hard Disk and select "Compact" from the "Utilities" pull down menu, a progress bar will appear to indicate the "Compacting virtual disk..." has started, again this will take some time, but at the end of the process the hard drive VMDK should have shrunk in size.



#7
besweeet

besweeet

    InsanelyMac Geek

  • Members
  • PipPipPip
  • 172 posts
  • Gender:Male

I use Disk Utility to "Erase Free Space", select your virtual hard disk in DU select the Erase tab and click on the "Erase Free Space..." button I leave the default setting of "Fastest" and click on the "Erase Free Space" button and wait for the Erasing Free space to complete, it will take some time. Once complete shutdown the OS X VM, and edit Virtual Machine Settings, select the Hard Disk and select "Compact" from the "Utilities" pull down menu, a progress bar will appear to indicate the "Compacting virtual disk..." has started, again this will take some time, but at the end of the process the hard drive VMDK should have shrunk in size.

I remember using Erase Free Space when the VMDK was at around 55GB, and it ended up filling the entire thing... I probably had the wrong setting. I'll try that the next time it fills itself up, thanks.



#8
besweeet

besweeet

    InsanelyMac Geek

  • Members
  • PipPipPip
  • 172 posts
  • Gender:Male
*massive bump*

Using WS11 and Yosemite, that command seems to do what it usually does, but after clicking on WS11's compact button for the VMDK, it instantly says that it's been finished while it doesn't actually do anything. Any ideas?

#9
Zenith432

Zenith432

    InsanelyMac Legend

  • Developers
  • 501 posts
  • Gender:Male
I have a worse problem which is that when I try to compact from vmware GUI, I get a popup with an error message "A required file was not found". I've searched the net for the popup, and it's mentioned but I couldn't find a solution.

What I do is compact the vmdk from the command line. Open a cmd window, and go to the directory where the vmware binaries are. Then run
vmware-vdiskmanager -k <full-path-to-vmdk>
When I use the command line version, it also prints a warning about a missing file, but then proceeds to compact the vmdk.

Go figure...
 

Using WS11 and Yosemite, that command seems to do what it usually does, but after clicking on WS11's compact button for the VMDK, it instantly says that it's been finished while it doesn't actually do anything. Any ideas?



#10
besweeet

besweeet

    InsanelyMac Geek

  • Members
  • PipPipPip
  • 172 posts
  • Gender:Male

I have a worse problem which is that when I try to compact from vmware GUI, I get a popup with an error message "A required file was not found". I've searched the net for the popup, and it's mentioned but I couldn't find a solution.

What I do is compact the vmdk from the command line. Open a cmd window, and go to the directory where the vmware binaries are. Then run

vmware-vdiskmanager -k <full-path-to-vmdk>
When I use the command line version, it also prints a warning about a missing file, but then proceeds to compact the vmdk.

Go figure...


That command worked for me, thanks.

#11
Donk

Donk

    InsanelyMac Deity

  • Members
  • PipPipPipPipPipPipPipPipPipPip
  • 2,096 posts
  • Gender:Male
  • Location:Manchester UK

*massive bump*

Using WS11 and Yosemite, that command seems to do what it usually does, but after clicking on WS11's compact button for the VMDK, it instantly says that it's been finished while it doesn't actually do anything. Any ideas?

 

Compact only seems to work for filing systems such as NTFS and FAT32. Does not work with Linux VMs.



#12
Zenith432

Zenith432

    InsanelyMac Legend

  • Developers
  • 501 posts
  • Gender:Male

You need to wipe the free space on the target filesystem to zeros.  For Linux, can use a bash command like in post #2 above.  It works for me with ext4.  I don't know of Linux tools to wipe free space other than the bash method.

 

The vmware compaction eliminates all-zero sectors from the vmdk sparse files.  That's why they need to be wiped first.

On Windows there's a tool called sdelete that can be used with -z switch to wipe free space to zero.  It's part of sysinternals suite.

 

There are also commercial tools like Active KillDisk, Jetico BCWipe that do a better job of wiping filesystem metadata.  The bash script only wipes free allocation blocks, but not metadata.  In OS X there's the "Erase Free Space" which MSoK mentioned above (post #6) - I think it also wipes only free allocation blocks, not metadata.

 

Compact only seems to work for filing systems such as NTFS and FAT32. Does not work with Linux VMs.



#13
Donk

Donk

    InsanelyMac Deity

  • Members
  • PipPipPipPipPipPipPipPipPipPip
  • 2,096 posts
  • Gender:Male
  • Location:Manchester UK

You need to wipe the free space on the target filesystem to zeros.  For Linux, can use a bash command like in post #2 above.  It works for me with ext4.  I don't know of Linux tools to wipe free space other than the bash method.

 

The vmware compaction eliminates all-zero sectors from the vmdk sparse files.  That's why they need to be wiped first.

On Windows there's a tool called sdelete that can be used with -z switch to wipe free space to zero.  It's part of sysinternals suite.

 

There are also commercial tools like Active KillDisk, Jetico BCWipe that do a better job of wiping filesystem metadata.  The bash script only wipes free allocation blocks, but not metadata.  In OS X there's the "Erase Free Space" which MSoK mentioned above (post #6) - I think it also wipes only free allocation blocks, not metadata.

 

Sorry I was mistakenly talking about something different. I actually was referring to the "Clean Up Disks" menu item, which I confused with the  compact options. You can still use VMware tools from the command line to do much of this.

 

Windows: VMwareToolboxCmd.exe

Linux: vmare-toolbox-cmd

OSX: vmware-tools-cli

 

Run the command with "help disk" to see the options.



#14
Zenith432

Zenith432

    InsanelyMac Legend

  • Developers
  • 501 posts
  • Gender:Male

I don't see any "Clean Up Disks".  I see "defragment" and "compact".

 

Not sure what "defragment" does.  I use the contig tool from sysinternals, and after "defragment" it reports the vmdk still fragmented as a file.  "defragment" refers to some vmdk internal fragmentation - and I don't know what's the benefit of it.  After doing "compact" - doing "defragment" says it's already defragmented.

 

Thanks for the tip about vmware-toolbox-cmd.  I'll try it.



#15
Donk

Donk

    InsanelyMac Deity

  • Members
  • PipPipPipPipPipPipPipPipPipPip
  • 2,096 posts
  • Gender:Male
  • Location:Manchester UK

I don't see any "Clean Up Disks".  I see "defragment" and "compact".

 

 

It's on the VM-->Manage menu in Workstation 10 and 11. Also similar functionality in Fusion on first tab of the Settings dialog.



#16
Zenith432

Zenith432

    InsanelyMac Legend

  • Developers
  • 501 posts
  • Gender:Male

Ok, I never noticed that "Clean Up Disks" :)  Have no idea what it does.  Just tells me that "no cleanup is necessary".

 

I tried wiping ext4 using vmware-toolbox-cmd, and then compacting - works, with result comparable to what I usually get with the bash command from post #2.  :thumbsup_anim:



#17
besweeet

besweeet

    InsanelyMac Geek

  • Members
  • PipPipPip
  • 172 posts
  • Gender:Male
Another bump...

I tried using vmware-vdiskmanager -k, but got the following:
VixDiskLib: Invalid configuration file parameter. Failed to read configuration file.
Failed to shrink the disk '[redacted].vmdk' : An error occurred while writing a file; the disk is full. Data has not been saved. Free some disk space and try again (0x8).
None of my physical drivers are full, nor is the VMDK. Unless it needs enough free space to match the existing VMDK size?

#18
justvisiting

justvisiting

    InsanelyMac Sage

  • Members
  • PipPipPipPipPip
  • 275 posts

I had enough local disk space, so I created another blank VM disk in vmware player, attached it to the  OSX virtual machine and used Carbon Copy Cloner to make an identical bootable copy. 

 

Once complete, I disconnected the original disk and verified the cloned disk booted successfully and that I could my regular tasks.

 

then I deleted the old vm disk.   

 

very fast on SSDs  :)



#19
besweeet

besweeet

    InsanelyMac Geek

  • Members
  • PipPipPip
  • 172 posts
  • Gender:Male

I had enough local disk space, so I created another blank VM disk in vmware player, attached it to the  OSX virtual machine and used Carbon Copy Cloner to make an identical bootable copy. 
 
Once complete, I disconnected the original disk and verified the cloned disk booted successfully and that I could my regular tasks.
 
then I deleted the old vm disk.   
 
very fast on SSDs   :)


That would be a decent alternative -- thanks.





0 user(s) are reading this topic

0 members, 0 guests, 0 anonymous users

© 2015 InsanelyMac  |   News  |   Forum  |   Downloads  |   OSx86 Wiki  |   Mac Netbook  |   PHP hosting by CatN  |   Designed by Ed Gain  |   Logo by irfan  |   Privacy Policy