459 replies to this topic

[Albert Nietsnie]

Good day.

I wrote a little program which allows to create a virtual machine and run any flavor of Mac OS X

• Mac OS X
  
• Mac OS X Server

inside the virtual machine, using any VMware product

• VMware Workstation for Windows
  
• VMware Workstation for Linux
  
• VMware Fusion for Mac OS X

on any Intel-based physical machine

• PC
  
• Mac

I hope you will enjoy it. For details, please read the file README.txt. Give me your feedback about it in this forum.

Attached Files

•  macosx_guest_vmware_7.tar.gz   52.25K   20749 downloads

[Albert Nietsnie]

People are already asking me why my solution is better than the current solution at http://www.insanelym...howtopic=172474 .

Here is why. My solution:

• Allows you to create Mac OS X virtual machines using the UI, without having to hand-edit the .vmx file.
  
• Does not require modifying VMware's darwin.iso.
  
• Will boot Mac OS X guest using the VMware's virtual EFI, which brings tons of improvements: nvram support, System Preferences > Startup Disk support, panic reporting support, Snow Leopard guests boot the 64-bit kernel by default, ...

[Donk]

Albert Nietsnie, on Jun 10 2010, 07:49 PM, said:

People are already asking me why my solution is better than the current solution at http://www.insanelym...howtopic=172474 .

Here is why. My solution:

• Allows you to create Mac OS X virtual machines using the UI, without having to hand-edit the .vmx file.
  
• Does not require modifying VMware's darwin.iso.
  
• Will boot Mac OS X guest using the VMware's virtual EFI, which brings tons of improvements: nvram support, System Preferences > Startup Disk support, panic reporting support, Snow Leopard guests boot the 64-bit kernel by default, ...

Interesting. Would you mind sharing what you have done?

[Albert Nietsnie]

I decompiled the vmware-vmx and vmware processes, and painstakingly determined what to change to remove all the restrictions VMware put in place.

Then I wrote code to do that automatically (split the binary into code sections, then section disassembly, then search for some patterns, then code replacement) on all platforms, in a way that hopefully will adapt when VMware releases new versions.

I purposedly do not want to give too many details (hence no source code), because I don't want VMware to figure out too easily what to change to make the life of my unlocker harder in future releases.

[Donk]

Albert Nietsnie, on Jun 10 2010, 08:48 PM, said:

I decompiled the vmware-vmx and vmware processes, and painstakingly determined what to change to remove all the restrictions VMware put in place.

Then I wrote code to do that automatically (split the binary into code sections, then section disassembly, then search for some patterns, then code replacement) on all platforms, in a way that hopefully will adapt when VMware releases new versions.

I purposedly do not want to give too many details (hence no source code), because I don't want VMware to figure out too easily what to change to make the life of my unlocker harder in future releases.

No problem. I had looked at the code myself, and realised where some of the areas that could be hacked were. I guess so long as the code remains in the product and the binary patches are kept up to date then a great solution. Just installing now. Found that existing VMs using my method cause the CPU to stop when using this method. After spending 2 and half years working on this, you have delivered something which means I can hopefully retire my solution.

One idea, this forum isn't the one most VMware users hang out in. Would you consider posting in the "Multiple Boot & Virtualization" forum?

Great work!!!

UPDATE: Got it working as I forgot to change the smc paramter to true in my exisiting guests, as you are obviously adding the OSK0/1 fields into the emulation. ;-)

[Albert Nietsnie]

Quote

I guess so long as the code remains in the product and the binary patches are kept up to date then a great solution.

Yeah I wondered about that. I think my heuristics to find the code/data to modify are quite solid, but only time will tell if the unlocker is going to be a maintenance nightmare or not.

Quote

One idea, this forum isn't the one most VMware users hang out in. Would you consider posting in the "Multiple Boot & Virtualization" forum?

I will post there and give a link to here (I don't want to have comments in 2 places, and the tarball refers to this topic's URL).

Quote

After spending 2 and half years working on this, you have delivered  something which means I can hopefully retire my solution. Great work!!!

Thanks. Given your creds here, it means a lot to me!

Quote

UPDATE: Got it working as I forgot to change the smc paramter to true in my exisiting guests, as you are obviously adding the OSK0/1 fields into the emulation. ;-)

Yes, otherwise the unlocker would not work on non-Apple hardware. I highly recommend you create a new VM (i.e. really the .vmx) using the UI, so you get all the proper settings (firmware = "efi" for example) in it, then you can attach an existing .vmdk to that new VM.

[Donk]

Albert Nietsnie, on Jun 10 2010, 09:13 PM, said:

Yeah I wondered about that. I think my heuristics to find the code/data to modify are quite solid, but only time will tell if the unlocker is going to be a maintenance nightmare or not.



I will post there and give a link to here (I don't want to have comments in 2 places, and the tarball refers to this topic's URL).



Thanks. Given your creds here, it means a lot to me!



Yes, otherwise the unlocker would not work on non-Apple hardware. I highly recommend you create a new VM (i.e. really the .vmx) using the UI, so you get all the proper settings (firmware = "efi" for example) in it, then you can attach an existing .vmdk to that new VM.

Got my guest working now. I had the VM already hacked up with the firmware = "efi" status as was looking at modding the BIOS. Already had it fixed for non-server version, but not easy to add an SMC emulator into the raw files inside it.

If ever want to have something tested then please PM me, as would be very happy to help out.

[Albert Nietsnie]

Quote

I had the VM already hacked up with the firmware = "efi" status as was looking at modding the BIOS. Already had it fixed for non-server version, but not easy to add an SMC emulator into the raw files inside it.

I thought about modding the virtual EFI code as well. But in the end I realized it was much easier to locate the "VM is not running Mac OS X Server. Will power off." string in the vmware-vmx binary, then locate the code which refers to the string, then modify the binary to nuke that code.

It is also a better solution, because it works both when booting from virtual BIOS and for when booting from virtual EFI

Quote

If ever want to have something tested then please PM me, as would be very happy to help out.

Thanks for the offer!

[hippoth]

Albert Nietsnie, on Jun 10 2010, 12:28 PM, said:

Good day.

I wrote a little program which allows to create a virtual machine and run any flavor of Mac OS X

• Mac OS X
  
• Mac OS X Server

inside the virtual machine, using any VMware product

• VMware Workstation for Windows
  
• VMware Workstation for Linux
  
• VMware Fusion for Mac OS X

on any Intel-based physical machine

• PC
  
• Mac

I hope you will enjoy it. For details, please read the file README.txt. Give me your feedback about it in this forum.

Great work, I will test this tomorrow at work.

Do you have plans to run this on ESXi?

[Albert Nietsnie]

Quote

Do you have plans to run this on ESXi?

I have never tested on ESXi because I know very little about ESXi (maybe I should just try running it in a VM). Would you like to help me to test it? I probably just need a few answers to extend the unlocker so it can also handle ESXi.

1) What is the ESXi environment like? Can you get a shell and run Linux executables?
2) If yes, how can you distinguish between regular Linux and ESXi? I.e. what is an easy way (using file operations) to identify ESXi?
3) If there are executables named vmware-vmx* or similar on the filesystem, in which directory are they and how are they named exactly?
4) Is the UI to create VMs web-based?

[Risner]

Donk, on Jun 10 2010, 03:52 PM, said:

After spending 2 and half years working on this, you have delivered something which means I can hopefully retire my solution

I hope not! ;-)  Your solution is the only one that I could use.  I don't personally know anyone
with Fusion or Workstation 7.  I do know at least 6 people with ESXi boxes at home, and I'm not counting
all the companies with ESXi boxes in production.

hippoth, on Jun 10 2010, 06:02 PM, said:

Do you have plans to run this on ESXi?

From the look of the tar file, this is implemented as a binary patch to the emulation binaries that run the VM's.
It would be similar (and even more scary) than the patching and resigning of the darwin.iso file.

Albert Nietsnie, since you haven't released code, I would assume there is no way to patch a Bios file or some other
"in the VM" way to accomplish this like what Donk's method uses?

As for not releasing the code, that is your choice.  But anyone wanting to know what you did will load it up in IDA pro and see.  Or use a decompiler.

Just so you don't think I'm a downer on your attempt.  I think it appears to be an elegant method.  I've use similar methods in the past on network protocol decoders for Everquest and more recently on locating moved code on iPhone internals in private API's.  I just won't ever be able to take advantage of this until it can (if ever) be done entirely in the BIOS (vmware's virtual bios) or EFI firmware.

[Albert Nietsnie]

So if I understand properly, you want the unlocker to support ESXi. Since you are the 2nd person to ask, I guess this is what I should implement next

Quote

As for not releasing the code, that is your choice.  But anyone wanting  to know what you did will load it up in IDA pro and see.  Or use a  decompiler.

Yes. They will have to expand the same amount of effort as VMware (or Apple) would.

Quote

From the look of the tar file, this is implemented as a binary  patch to the emulation binaries that run the VM's. It would be similar  (and even more scary) than the patching and resigning of the darwin.iso  file. I would assume there is no way to patch a Bios file or some other
"in the VM" way to accomplish this like what Donk's method uses? I just won't ever be able to take advantage of this until it can  (if ever) be done entirely in the BIOS (vmware's virtual bios) or EFI  firmware.

Yes. That is the fundamental difference between Donk's method and mine. Donk's method is "in the VM". Mine is "out of the VM".

Why won't you be able to take advantage of this? Because you are not allowed to run modified ESXi bits? I think you will learn to trust me over time, when you realize that my mods are surgical and that the modded ESXi runs just as fine as the unmodded one

[04e46conv]

Thanks for posting! I am install 10.6 now. Is it safe to run updates? Any do's and donts?

[Risner]

Albert Nietsnie, on Jun 10 2010, 06:45 PM, said:

Yes. They will have to expand the same amount of effort as VMware (or Apple) would.

Why won't you be able to take advantage of this?

modded ESXi runs just as fine as the unmodded one

With the right tools, it takes around 2 minutes.  I've got IDA Pro ;-)

Most of the places I'd like to put a VM like this are on ESXi boxes that run other VM's.  There is too much risk in running modified code (even Donk's was an issue for me) and the risk associated with a Vmware patch replacing the binary and requiring a reinstall of your patch.

I guess the only way to properly do this is to dedicate a system to just Snow Leopard VM's.  I've read about running Workstation in a VM, but I don't remember if that was "it works" or "it can't work."  I know you can run ESXi in an ESXi VM, can you run Workstation in an ESXi VM?  If so, it might be worth buying a Workstation license per Snow Leopard to do this.

Quote

Nested VMs are only possible when the host is running ESX 3.0 (with some caveats), ESX 3.5 (some AMD CPUs only), Workstation 6.0 (or later), or Fusion. Note that you cannot run nested VMs on Intel systems when the host is running ESX 3.5.

Kills the idea of running Workstation inside an ESXi, since I only have Intel CPU's.  ;-)

[scotia]

Albert Nietsnie, on Jun 11 2010, 08:16 AM, said:

I have never tested on ESXi because I know very little about ESXi (maybe I should just try running it in a VM). Would you like to help me to test it? I probably just need a few answers to extend the unlocker so it can also handle ESXi.

1) What is the ESXi environment like? Can you get a shell and run Linux executables?
2) If yes, how can you distinguish between regular Linux and ESXi? I.e. what is an easy way (using file operations) to identify ESXi?
3) If there are executables named vmware-vmx* or similar on the filesystem, in which directory are they and how are they named exactly?
4) Is the UI to create VMs web-based?

Hi Albert,

I'm keen to run this on an ESXi server, although I would like to continue running my 8 or so FreeBSD and Windows VMs on the same server.  I haven't read your instructions fully so I'm not sure if there are implications running other OSes alongside OSX.

To answer your questions above:

My ESXi environment is running from a USB Flash stick, which is installed by pulling out the disk image that comes as part of the ESXi installer.  That image is then dd'd onto the flash and hey presto ESXi running from flash.

Here are the instructions for extracting the image (using OSX):
http://blog.scottlow...ck-on-mac-os-x/

ESXi runs a pared down linux kernel with just a handful of drivers and userland utilities.  ESX proper (note there is no 'i') runs a full Linux installation from what I understand.  I think a lot of people run ESXi because of it's small footprint (can run from flash) and it's free.

Once installed you can enable the console on ESXi, which gives you a shell.  From there you can enable SSH access.

~ # uname -a
VMkernel esx-01.thismonkey.com 4.0.0 #1 SMP Release build-208167 Nov  8 2009 01:02:11 x86_64 unknown

Here is a dump of a VM directory:

~ # ls -al /vmfs/volumes/datastore-01/VMware-RCLI-3.5-U2-Appliance/
drwxr-xr-x	1 root	 root			   2100 Feb 14 01:29 .
drwxr-xr-t	1 root	 root			   1960 May 17 15:56 ..
-rw-------	1 root	 root		 1073741824 Feb 13 08:20 VMware-RCLI-3.5-U2-Appliance-flat.vmdk
-rw-------	1 root	 root			   8684 Feb 13 08:20 VMware-RCLI-3.5-U2-Appliance.nvram
-rw-------	1 root	 root				497 Feb 14 01:29 VMware-RCLI-3.5-U2-Appliance.vmdk
-rw-------	1 root	 root				720 Jun  4  2009 VMware-RCLI-3.5-U2-Appliance.vmsd
-rwxr-xr-x	1 root	 root			   3041 Feb 14 01:29 VMware-RCLI-3.5-U2-Appliance.vmx
-rw-------	1 root	 root				283 Feb 13 07:46 VMware-RCLI-3.5-U2-Appliance.vmxf
-rw-------	1 root	 root		 1073741824 Jan 26  2009 VMware-RCLI-3.5-U2-Appliance_1-flat.vmdk
-rw-------	1 root	 root				444 Feb 14 01:29 VMware-RCLI-3.5-U2-Appliance_1.vmdk
-rw-------	1 root	 root		 2147483648 Feb 13 08:20 VMware-RCLI-3.5-U2-Appliance_2-flat.vmdk
-rw-------	1 root	 root				499 Feb 14 01:29 VMware-RCLI-3.5-U2-Appliance_2.vmdk
-rw-r--r--	1 root	 root			1083758 Jan 28  2009 vmware-1.log
-rw-r--r--	1 root	 root			  40804 Feb 13 08:20 vmware-2.log
-rw-r--r--	1 root	 root			  24543 Feb 14 01:29 vmware.log

The VMs are pretty much interchangable between VMWare products.  Today I SSH'd a copy of Windows Server 2008 which I had running under Fusion to my ESXi box and it fired up nicely (after changing the NIC driver).

Finally, VMs are created using vSphere Client which is a Windows program you need to install (elsewhere obviously).

I was thinking that the best place to patch the ESXi binaries would be on the disk image BEFORE it's dd'd over to flash.  This way you can use either OSX or Windows to launch the patcher from.

Thanks and good luck.  I'm happy to test if you need it.
Scott

[Risner]

Albert, I missed your question or I would have answered them.

scotia, on Jun 10 2010, 11:53 PM, said:

My ESXi environment is running from a USB Flash
That image is then dd'd onto the flash and hey presto ESXi running from flash.

ESXi runs a pared down linux kernel with just a handful of drivers and userland utilities.
ESX proper (note there is no 'i') runs a full Linux installation from what I understand.

I was thinking that the best place to patch the ESXi binaries would be on the disk image BEFORE it's dd'd over to flash.

A couple comments, current ESXi is installed to flash via booting the ISO and selected the flash as the install disk.  So the /bin/dd method is deprecated.

VMware somewhere announced that ESX is going away in the future and all paid & free ESX will be ESXi.  Current ESXi 4 is both paid and free, you unlock features by paying the license fee and entering the license (one prime example is jumbo frames on NAS.)

File /etc/vmware/ft-vmk-version has the version:
product-version = 4.0.0
ft-version = 193498

Compiling apps to run on ESXi can be done, but there are a LOT of libs you can't use (ncurses for example is problematic.)  You also need to link it statically.  I've got rsync and a few other apps built to run native on the ESXi systems.

# find / -name \*vmware-vmx\* -print
#
So no vmware-vmx executable.

Each vm does have a proc running:
# ls -l /bin/vmx
-rwsr-xr-x    1 root     root            8690492 Sep 17  2009 /bin/vmx
It appears to fork off or is ran multiple times (I've never checked which) because
each VM seems to have multiple processes of this binary.

UI is web based, in a round about way I believe (again never checked) since the .Net app (Windows only) used to manage I believe does all it's work over an https connection to the server.

[Donk]

Risner, on Jun 11 2010, 05:12 AM, said:

Albert, I missed your question or I would have answered them.



A couple comments, current ESXi is installed to flash via booting the ISO and selected the flash as the install disk.  So the /bin/dd method is deprecated.

VMware somewhere announced that ESX is going away in the future and all paid & free ESX will be ESXi.  Current ESXi 4 is both paid and free, you unlock features by paying the license fee and entering the license (one prime example is jumbo frames on NAS.)

File /etc/vmware/ft-vmk-version has the version:
product-version = 4.0.0
ft-version = 193498

Compiling apps to run on ESXi can be done, but there are a LOT of libs you can't use (ncurses for example is problematic.)  You also need to link it statically.  I've got rsync and a few other apps built to run native on the ESXi systems.

# find / -name \*vmware-vmx\* -print
#
So no vmware-vmx executable.

Each vm does have a proc running:
# ls -l /bin/vmx
-rwsr-xr-x    1 root     root            8690492 Sep 17  2009 /bin/vmx
It appears to fork off or is ran multiple times (I've never checked which) because
each VM seems to have multiple processes of this binary.

UI is web based, in a round about way I believe (again never checked) since the .Net app (Windows only) used to manage I believe does all it's work over an https connection to the server.

Just to add to this good post.

1. ESX and ESXi 4.0 are not based on Linux which is a popular misconception. The hypervisor is it's own OS. It is true there are 2 different layers of Linux capabilities inside ESX(i).

• ESX - the console operating system is based on Redhat Linux - but from ESX 4 is not used to boot the system. It is in essence a privileged guest OS that resides inside its own VMDK. Similar to Dom0 in Xen
  
• ESXi - the command line system is based on BusyBox so utilities would need to be compiled for that environment
  
• ESX & ESXi - the driver level (via the DDK) uses a Linux emulation API to aid in driver porting. The open source code can be obtained from VMware, but it is pretty hard to get a decent environment to build unless you are part of the paid VMware Partner programme.

2. The vmware-vmx equivalent is the vmx process mentioned in the previous post. However it will not have all the hooks you have found for the latest hosted products. ESX tends to run one version behind the core virtual platform inside the hosted products. So:

• VMware Workstation 7.x, Player 3.x & Fusion 3.x are related - Include newer HPET code and EFI BIOS for booting Darwin.
  
• VMware Workstation 6.5, Player 2.5 and Fusion 2.5 are related to ESX(i) 4 - and does not contain the EFI BIOS capabilities only the darwin.iso method for booting Darwin.

3. The front-end for ESXi free & paid edition can ignore the parameters and fire up a Mac OS X guest, the addition of vCenter server stops this. So you have to control the ESX host via a direct connection. The front end code is a mix of Windows C# and C++ code talking through a SOAP API to the hostd process running on the ESX(i) box. That means to add a seamless experience to the product means patching in many different areas.

4. As the virtual hardware lags behind Workstation 7 et al., my latest darwin.iso patches the kernel in memory before launching it. Primarily the LAPIC version is too low for the 10.6 kernel, so a patch (done with Meklort from Chameleon team) removes the panic to NOPs. Also issues with FSB reporting using the VMware boot-132 code, which Chameleon fixes. (Code available if you want it as per APSL. Was waiting to get to a final version before releasing it.)

A pretty big challenge! Not to say it isn't worth trying.

Risner, on Jun 11 2010, 04:19 AM, said:

I guess the only way to properly do this is to dedicate a system to just Snow Leopard VM's.  I've read about running Workstation in a VM, but I don't remember if that was "it works" or "it can't work."  I know you can run ESXi in an ESXi VM, can you run Workstation in an ESXi VM?  If so, it might be worth buying a Workstation license per Snow Leopard to do this.



Kills the idea of running Workstation inside an ESXi, since I only have Intel CPU's.  ;-)

You can run Workstation inside ESXi 4 with Intel CPUs but it won't give you what you want.  You are limited to 32-bit nested guests with no access to VT-x, which is essential for the built-in Mac OS X virtualization.

(Funny this, I was the first one to post how to do this on VMware forums with the help of Jim Matteson, one of the VMware engineers!)

[Albert Nietsnie]

Quote

I am install 10.6 now. Is it safe to run updates? Any do's and donts?

If your physical CPU is a Core i5 or Core i7 and you are running 10.6 guest, do _not_ run Software Update in the VM right now. Wait until Apple releases 10.6.4 (a matter of days I believe), then only you can safely run Software Update in the VM.

Quote

I haven't read your instructions fully so I'm not sure if there are implications running other OSes alongside OSX.

There are no implications running other OSes alongside OSX.

Quote

I was thinking that the best place to patch the ESXi binaries would be on the disk image BEFORE it's dd'd over to flash.  This way you can use either OSX or Windows to launch the patcher from.

Thanks for the details on how ESX(i) works. Why not launch the patcher from the console's shell?

Quote

I'm happy to test if you need it.

Great. I'll try to cook up something in the next few days, as time permits. I have a day job too!

Quote

Albert, I missed your question or I would have answered them.

Thanks for your details on ESXi.

The /bin/vmx binary is probably what I'm looking for.

My unlocker does not depend on any lib beside basic Posix (open, mmap, printf, ...) functionality, so compiling it for ESXi should not be an issue.

[Donk]

Albert Nietsnie, on Jun 11 2010, 09:26 AM, said:

If your physical CPU is a Core i5 or Core i7 and you are running 10.6 guest, do _not_ run Software Update in the VM right now. Wait until Apple releases 10.6.4 (a matter of days I believe), then only you can safely run Software Update in the VM.


There are no implications running other OSes alongside OSX.


Thanks for the details on how ESX(i) works. Why not launch the patcher from the console's shell?


Great. I'll try to cook up something in the next few days, as time permits. I have a day job too!


Thanks for your details on ESXi.

The /bin/vmx binary is probably what I'm looking for.

My unlocker does not depend on any lib beside basic Posix (open, mmap, printf, ...) functionality, so compiling it for ESXi should not be an issue.

You should be OK from the console shell.

One suggestion; how about backing up the files before patching then allow them to be  copied back if needed to undo the patch?

[Albert Nietsnie]

Quote

ESXi - the command line system is based on BusyBox so utilities would need to be compiled for that environment

OK I'll build the ESXi unlocker against that environment.

Quote

The vmware-vmx equivalent is the vmx process mentioned in the previous post. However it will not have all the hooks you have found for the latest hosted products. ESX tends to run one version behind the core virtual platform inside the hosted products.

It seems that version 4.1 of ESXi is going to be released soon. Hopefully it will have the same virtual hardware as Workstation 7/Fusion 3. Maybe I should just wait for that and see.