This was originally a post asking for the definitive guide. Since I did not get a response, I continued tinkering. I eventually was able to patch something together based on other tutorials. Below what I have done.
If you run into issues or mistakes from my end, or certain steps don’t turn out to be necessary, please comment below so we can make this guide complete!
ESXi 6.7 installed and working
My hardware: Fujitsu D3417-B2 (Intel C236), Xeon E3-1245 v6, 32GB RAM, 1250 GB SSDs and 8 TB HDDs
This guide assumes that all steps are executed on a mac; most can be done on most other operating systems as well, except for Step 3.
Installing fully functional version of macOS Mojave on ESXi. Why would one want this? I upgraded my server and installed ESXi (a virtualisation hypervisor) to run several operating systems side by side. Since the server is now located near my desk, I wanted to use the opportunity to also use that system as a workstation for some of my photo hobby stuff, rather than using my work MBP.
Install ESXi (not covered here)
Create a bootable ISO
Create a new virtual machine
Install VMware Tools
Unlocker. Latest version verified for ESXi is v.3.0.0 (https://www.insanelymac.com/forum/files/file/964-macos-unlocker-v30-for-vmware-esxi/ ; credits to @Donk; you need to be logged in to download)
MacOS Mojave (See instructions below. For this, you need a macOS machine)
VMware Tools for macOS (https://www.insanelymac.com/forum/files/file/31-vmware-tools-for-os-x-darwiniso/ ; credits to @MSoK; you need to be logged in to download)
Step 2: Install Unlocker
Normally, macOS can only be installed on Apple hardware. The unlocker will modify ESXi to accept non-Apple hardware for macOS VMs.
Using the link above, download Unlocker and upload the file to your ESXi systems
Enable SSH in ESXi
Open a terminal window
SSH into ESXi (“ssh login_name@ESXi_IP”)
Navigate to the folder where you uploaded the unlocker (datastores are located in “/vmfs/volumes/“
Unpack the Unlocker (“tar xzvf esxi-unlocker-xxx.tgz”)
You may need to make the script executable. This can by done by typing “sudo chmod -x esxi-install.sh” (you will likely be asked for your password)
Run the script using “./esxi-install.sh”
Step 3a: Download MacOS Mojave
This is easy if you have a MacOS machine. There are other sources, but I would not recommend using them for security reasons
Within the Mac App Store find MacOS Mojave: https://itunes.apple.com/us/app/macos-mojave/id1398502828?ls=1&mt=12
Click “Get” and the update process starts
Depending on the version of macOS installed, the update process will automatically halt after download or you will need to press CMD+Q as soon as the download is finished
As a result you will find a new ‘app’ in your applications folder: “Install macOS Mojave.app”
Step 3b: Create a bootable ISO
From the file above, we need to create a bootable ISO. Bill Plein made an excellent script for this that you can use one-on-one: https://blog.plein.org/2018/09/28/creating-a-macos-10-14-mojave-installation-iso/
Open the Textedit app and create a new file
In the app’s preferences choose ‘Plain text’ as the document format
Paste the script and save as “mk-mojave-iso.sh”
Open a Terminal window and navigate to the folder with the script
You may need to make the script executable. This can by done by typing “sudo chmod -x mk-mojave-iso.sh” (you will likely be asked for your password)
Then run the script: “./mk-mojave-iso.sh”
The script will run for a couple of minutes and as a result a file named “Mojave.iso” will be placed on your desktop
Upload the file to ESXi
NB: the file will be stored on your Desktop and hence possibly automatically synced with your iCloud, make sure to remove/delete it to save iCloud space
Step 4: Create a new virtual machine
Log into the ESXi web interface
From the Virtual Machines section click “Create / Register VM”
Click “Create a new virtual machine”
Give it whatever name you like
Choose “Mac OS” as the guest OS
Choose “Apple Mac OS X 10.14 (64-bit)” as the guest OS version
Choose the datastore you want to use
I opted to use 2 cores, 100GB and 4GB in RAM
Make sure to give the VM a unique MAC address by setting it to manual under Network Adapter 1 (I took the server’s MAC address and added ‘1’ to the last digit)
I chose adapter type E1000e for no particular reason, other than that one was required in one of my other OSs
As CD/DVD Drive 1, choose Datastore ISO file, make sure “Connect at power on” is checked and choose the bootable ISO that you uploaded
Click “Finish” to create the VM
Step 5: Install MacOS
Run the VM you have just created
If everything worked correctly, the VM will boot from the bootable ISO (The screen will say it is “attempting to start up from … Virtual SATA CDROM Drive (1.0)”). If not, the ISO file was likely not created correctly (Step 3) or it was not properly mounted in Step 4.
You will see the white Apple logo and a progress bar
You will be asked to select your language
Then you will enter the setup menu, choose “Disk Utility”
Select the “VMware Virtual S..” on the left and choose “Erase”, this will format your virtual drive. You can give it any name you want (I went traditional with “Macintosh HD”, I chose “Mac OS Extended (Journaled)” as the format and “GUID Partition Map” as the scheme).
When formatted, you can continue the installation process (this mostly speaks for itself). You are asked several questions. I opted out of all things that would send data to Apple (location services, Apple ID) as I do not want to load their infrastructure with my system. Please share whether you succeeded if you have chosen otherwise.
You should now be able to boot into a working version of MacOS
Step 6: Install VMware Tools
Finally, it is recommended to install VMware Tools. More info: https://www.vmware.com/support/ws5/doc/new_guest_tools_ws.html. It can be downloaded here: https://www.insanelymac.com/forum/files/file/31-vmware-tools-for-os-x-darwiniso/ (com.vmware.fusion.tools.darwin.zip.tar; extract and find darwin.iso)
Upload Darwin.iso to an ESXi datastore
Shut down the running MacOS VM (if you hadn’t done so already)
Edit the settings of the VM and navigate to the CD/DVD drive: replace the MacOS bootable ISO with the Darwin.iso file
Save and start the MacOS VM
When started, the darwin.iso should be mounted and you should be able to install VMware Tools
To tinker together the above, I used (amongst others) the below:
https://ithinkvirtual.com/2017/02/12/create-macos-os-x-vm-on-vmware-esxi-6-5-vmware-workstation-12-x/ (older version of macOS and ESXi)
https://www.youtube.com/watch?v=RM_YcTbsC8k (this one uses modified files from a service I do not know one can trust)
Open questions / TODO’s
How to realise video passthrough?
I hope this is helpful; please share your experience so we can refine this guide if needed!