ESXi 5.5 with a 2GB Radeon 7750 in GPU passthrough

[ferebee]

I have a Supermicro server with a X8DTH-iF board, dual E5649 CPUs and 128 GB RAM that is mostly idle during the day, so I want to run ESXi to virtualize the server side and add a Mavericks VM to use as a desktop. So far, Mavericks is running fine under ESXi 5.5 (currently U1) with Unlocker 1.2.0, installed from  the App Store 10.9.2 download using the create_vmware_osx_install_dmg.sh script.

 

I’ve added a PowerColor Radeon HD 7750 Eyefinity4 2GB GPU and am trying to enable GPU passthrough. (This was the only card I could find that was low-profile, high-performance, and has Mini-DisplayPort outputs.) It shows up as a "Radeon 7xxx".

 

To use more than 2GB RAM in the VM, I have to set the pciHole parameters, currently at  pciHole.start = 1200 and pciHole.end = 2200. [Edit: I have since changed pciHole.end to 3200, though I wish I knew how to determine the proper setting except by guesswork.]

 

Everything seems to work fine at 1080p resolution, e. g. if I connect my DELL U2713HM with a Mini-DisplayPort to HDMI cable.

 

However, if I use a Mini-DP to DP cable, the DELL asks for 1440p (2560 x 1440) and Mavericks 10.9.2 kernel panics, apparently somewhere in the Radeon driver.

 

I’ve also tried replacing AMD7000Controller.kext, AMDRadeonX4000.kext, and AMDRadeonX4000GLDriver.bundle with the ones from 10.9.3 build 13D12, but the same thing happens.

 

Is this likely to be an issue with ESXI and PCI passthrough, or is this a problem with Mavericks vs. the 7750?

 

Thanks for any ideas, and for the great information here that helped me get this far in the first place!

[ferebee]

Update: IT WORKS!

 

I just replaced all of the kexts listed below with new ones from the latest beta of OS X 10.9.3 (build 13D38), leaving the rest of the system at 10.9.2, and am now able to select 2560 x 1440 resolution without a kernel panic. The card still shows up as an "AMD Radeon HD 7xxx 2048 MB", and I don`t think I have proper acceleration, but it`s usable.

 

Again, this is using pciHole.start = 1200 and pciHole.end = 3200. I haven`t tried multi-monitor support yet - the board has 4 Mini-DisplayPort connectors.

 

Here’s what I replaced:

 

AMD7000Controller.kext, AMDRadeonX4000.kext, AMDFramebuffer.kext, AMDRadeonX4000GLDriver.bundle, AMDRadeonVADriver.bundle, AMDSupport.kext

 

Merely replacing AMD7000Controller.kext and AMDRadeonX4000.kext was not sufficient, I’d still get a kernel panic when plugging the DELL 27" in via DisplayPort.

 

The next problem to solve will be input. At the moment I`m using Synergy from another Mac, which works well, but I hope to add a Fresco Logic FL1100 USB 3.0 card and pass it through to the VM, so I can use either direct USB input devices or a Bluetooth dongle. ESXi is not able to pass USB input devices to VMs as far as I can see, and while it does support virtualized Bluetooth, all the keyboards and mice I`ve tried appear to disconnect after a minute or two. From what I`ve read, PCI passthrough seems to work well with USB cards, and I gather OSX supports the FL1100 OOB.

[ferebee]

Updated to AMD kexts from 10.9.3 build 13D55. The GPU still reports as an "AMD Radeon HD 7xxx 2048 MB", but QE/CI are enabled as confirmed by DVD Player -> Help -> Show Supported Features. This was probably the case all along, I was just confused by the slightly jerky cursor motion that results from controlling the mouse via VNC (i. e., Synergy) instead of USB.

 

Turning off the monitor causes the GPU to shut down. OS X continues to run using the VMware SVGA display, but the Monitors control panel no longer shows a second display. To get the Radon display back, I have to reboot OS X. However, OS X is able to sleep the monitor and will wake it up again correctly. I haven't attempted to sleep/wake OS X itself.

 

I tried an Orico 2-port USB 3.0 card with FL1100 chip, but even using a fresh 10.9.2 VM and adding the Orico card to it as the sole PCI passthrough device causes a kernel panic in OS X. I've ordered a different card and will try that next with the GenericUSBXHCI.kext.

[ferebee]

Here's some more info, in response to a private message. Might as well post this in the forum for others...

 

The GPU I'm using is a PowerColor Radeon HD 7750 Eyefinity4 with 2 GB and 4 Mini-DisplayPort outputs. Since I'm running it in a Supermicro SC847 chassis, I needed a low-profile card, and since the server is sitting in the basement (noise!) and the video cable runs through a hole in the floor, I wanted Mini-DP because it’s the smallest connector. This was the only card I could find that fit all the requirements. It ships with adapters from Mini-DP to DP and Mini-DP to Single-Link DVI (i. e., max. 1080p).

 

The card magically started working in one of the later betas of 10.9.3, see my earlier posts, and runs OOB with no kext hacks.

 

Aside from OS X, I'm running SmartOS (a Solaris derivative) as a guest. It didn't seem to be too happy with the E1000 emulation provided by ESXi, so I currently have a supported Intel 1GbE card in passthru to SmartOS. There are also two LSI 9211-8i-IT controllers in passthru.

 

Something has not been quite right, unfortunately, as the SmartOS VM has been stalling out every few weeks. I haven't been able to debug it, but my impression is that disk I/O is stalled. If I have shells open to the SmartOS global zone or any its zones, it still takes text input, but nothing else happens. Scrubs of the ZFS file system never show any errors on reboot, but my Time Machine backups via netatalk tend to get damaged if they are running when a stall happens.

 

I  can reboot the SmartOS VM via ESXi, but then the problem appears to recur fairly quickly. Rebooting the entire ESXi host clears the problem, and things will be fine for weeks afterwards.

 

It feels like a poor interaction between ESXi, PCI passthru and SmartOS. The OS X VM is not affected by the stalls.

 

I have since tried two different USB 3.0 cards via passthru for OS X: the Orico Fresco Logic FL1100 that I thought would work OOB, and a VIA that I was trying to use via the GenericUSBXHCI kext. Neither worked (KP with the FL, dead USB ports on the VIA), but I didn't pull the VIA until yesterday. Part of the problem may have been that I did not connect auxiliary power to the USB cards, though I did only want to connect a self-powered hub.

 

Perhaps the USB cards were causing issues for ESXi.

 

I've also been thinking about making use of 10GbE, since there is no VMXNET3 driver for VMware OS X guests or SmartOS (though it might be possible to get the open-source Solaris driver to work), and the VMware E1000 emulation is limited to 1 Gbit/s throughput. After thinking about it, I hope to do something like this:

 

- one Intel X520-DP1 card in passthru to SmartOS

- one Intel X520-DP1 card configured in ESXi

- crossover connection between the two with an SFP+ direct attach cable

- motherboard Intel GbE uplink to the rest of the LAN

 

This would allow me to create iSCSI volumes on the SmartOS ZFS and provide them to ESXi as a VM backing store for my OS X volumes, giving high performance local storage to my Mac VM with the ability to do ZFS snapshots and backups as protection against HFS+ corruption.

 

Alternatively, I would put a Small Tree or ATTO 10GbE card in passthru to OS X and store my Mac data on a fast netatalk AFP share on SmartOS, but the cost for those 10GbE cards is really just a bit too much at this stage. Small Tree and ATTO sell re-branded Intel X520 and X540 cards with custom firmware that enables their Mac drivers, so you're mostly paying for the drivers. Things should change once Apple ships 10GbE, which I would expect to see sometime around Intel's next 10GbE chip revision.

 

None of this makes sense until I can cure the SmartOS hangs. Onwards!

[cotarelo]

Hello!

 

Amazing work! I was wondering whether you could use that Graphic card for more than 1 virtual machine at the same time?

 

Is the passthrough operating system dependent? I mean, could this solution stop working if tomorrow Apple will release OSX 10.12?