Jump to content

Graphics Acceleration in ESXi 5.1 and OS X Mountain Lion VM


  • Please log in to reply
23 replies to this topic

#1
sgreszcz

sgreszcz

    InsanelyMac Protégé

  • Members
  • Pip
  • 13 posts
I have a couple of Mac Minis (core 2 duo) which I have been using for video projects and transcoding of my kid's DVDs and old analogue home movies into H.264. Even with queuing up transcoding, it can take quite a while to get things right so I have been experimenting with getting Mountain Lion running on my 8-core ESXi 5.1 lab box. The installation went smoothly, thanks to everyone in this forum and for those that created the Unocker!

When I have tried to install my video software I get error messages saying that I need video accelerated hardware to be present. I am running ESXi on an ASUS Z8NA-D6 server, which has basic onboard VGA video (aspeed ast2050) which I believe is not accelerated in any means.

Is it just a matter of finding a basic hardware accelerated video card that is supported by both ESXi and OS X? If so, does anyone have any experience in this and can recommend an inexpensive one that would do the job?

Thanks in advance!

#2
Donk

Donk

    InsanelyMac Deity

  • Members
  • PipPipPipPipPipPipPipPipPipPip
  • 1,924 posts
  • Gender:Male
  • Location:Manchester UK
VMware have no drivers which run QE/CI needed for video acceleration. The only possible solution is to use VT-d to pass through an OS X supported video card. You will have to check that the motherboard, BIOS and CPU support VT-d. Note this is different form VT-x.

Update: It looks like it does support VT-d so could be in luck. And this thread may help you find one.

http://www.insanelym...n-asus-z8na-d6/

#3
sgreszcz

sgreszcz

    InsanelyMac Protégé

  • Members
  • Pip
  • 13 posts
Thanks For the reply Donk and for the excellent work on the unlocker. The installation went much smoother than expected.

I did some more research tonight and found out that luckily (as it is an older motherboard) that it supports VT-d.

I've ordered a NVIDIA GeForce GT640 which should be natively supported by mountain lion, so hopefully it will work in passthrough mode. Then I'll just have to figure out how to get sound working as I don't believe I have a sound card, USB or otherwise on the motherboard.

I'll report back with my findings and hopefully this goes easier than trying this the bare metal installation way.



#4
sebus

sebus

    InsanelyMac Sage

  • Members
  • PipPipPipPipPip
  • 261 posts
  • Gender:Male
  • Location:UK
Please do report back, as it looks like an interesting option

#5
sgreszcz

sgreszcz

    InsanelyMac Protégé

  • Members
  • Pip
  • 13 posts
OK, here is an update of the progress that I've made so far with installing Mountain Lion on my ESXi 5.1 box with GPU passthrough for accelerated graphics.

I spent a lot of time looking at these sites and searching with Google to find the most inexpensive and compatible out of box video card for Mountain Lion:

http://www.macbreake...hics-cards.html
######

I originally bought a nvidia GT 640, as it looked to be quite compatible with Mountain Lion, but in the end I found out that ESXi passthrough with nvidia cards did not work too well:

http://networkingbun...ts/vmdirectpath

So I returned the nvidia and spent time researching AMD Radeon cards here:

####

I found a Radeon HD6570 for about £40 and set up GPU passthrough in ESXi (I forgot to say that I did install Mountain Lion first without the Radeon card - I don't know if that would make any difference).

From your host, select the configuration tab, then "Advanced Settings", then "Edit", and select the ATI card.

Posted Image

Then you have to reboot your ESXi host for passthrough to take effect. You then can add the GPU to the OSX guest using the "Add PCI device" option. To allow the OSX guest to more memory, you need to adjust some settings on the host:


pciHole.start = "1200" and pciHole.end = "2200"


(Options/Advanced/General/Configuration Parameters) - in properties of VM.



I then had some errors when starting up the guest when I boosted the memory from 2GB to 16GB:




Failed to start the virtual machine.
Module MemSched power on failed.
An error occurred while parsing scheduler-specific configuration parameters.
Invalid memory setting: memory reservation (sched.mem.min) should be equal to memsize(xxxxx).

So to fix that I just clicked on the GPU PCI card that I added and clicked the option to "make reserved memory equal" for the OSX guest.

So after booting up, I was able to get all 16 virtual cores running, run Final Cut which didn't run before, and see the ATI card in system info:

Posted Image

I am using screen sharing from my Macbook Pro (also running Mountain Lion) to control the mouse and keyboard. I still haven't found a way to disable the VMware display, so I sort of get two of them, which I don't really want.

Posted Image

I also need to do some experimentation to see how well video editing and audio work, but at least now I can run programs that need video acceleration.

Edited by ErmaC, 08 April 2013 - 01:20 PM.
PLS read the forum rules. link removed


#6
Donk

Donk

    InsanelyMac Deity

  • Members
  • PipPipPipPipPipPipPipPipPipPip
  • 1,924 posts
  • Gender:Male
  • Location:Manchester UK
Brilliant write-up. Thanks for sharing the information with us.

#7
sebus

sebus

    InsanelyMac Sage

  • Members
  • PipPipPipPipPip
  • 261 posts
  • Gender:Male
  • Location:UK
Excellent, thanks for this, really clever way to do it

And what CPU do you run?

sebus

#8
sgreszcz

sgreszcz

    InsanelyMac Protégé

  • Members
  • Pip
  • 13 posts

And what CPU do you run?


2 x Intel Xeon E5620 Processor

#9
wxe911

wxe911

    InsanelyMac Protégé

  • Members
  • Pip
  • 17 posts
@ sgreszcz
First thanks for the detailed post and of course your research to get things to work.

I could not find any hint as to how you got sound to work?

Would you care to reveal your secret on that?

Regards, WXE911

#10
sgreszcz

sgreszcz

    InsanelyMac Protégé

  • Members
  • Pip
  • 13 posts

@ sgreszcz
First thanks for the detailed post and of course your research to get things to work.

I could not find any hint as to how you got sound to work?

Would you care to reveal your secret on that?

Regards, WXE911


Well, sound is a work in progress, and I still don't have it sorted out yet.

I installed Soundflower via a screen capture app which gave me some sound (albeit choppy for some reason - maybe as it was in software?) so I started to look for hardware solutions.

I tried to passthrough the USB hardware and use a USB dongle for sound. The USB dongle works natively on my Macbook, without needing drivers, but there is a bug in ESXi5.1 that allows you to passthrough the USB hardware, but after rebooting the host server to enable it, the USB device disappears.

[Seems like VMware removed USB passthrough from 5.1 for some reason when it used to work in 5.0]

I then tried to add the USB sound dongle as a virtual USB device. I couldn't get that to work either - The guest OSX didn't detect it.

I then bought the M-Audio Audiophile 2496 which is a PCI card with older, but supposedly native drivers for OSX. I couldn't get the PCI card to passthrough. I could select it, but when I rebooted the ESXi host, I couldn't get Mountain Lion to boot.

I have now ordered a Maya 44 XTe PCI Express Audio Interface to see if I can passthrough that better (as I was able to passthrough the Radeon PCI Express card). It is a bit pricey, but has native Mountain Lion drivers supported by the manufacturer.

I'll report back when I receive the card and have time to test it. I think that this will be my last attempt. If it doesn't work, I might give up on using the ESXi/OSX server as a workstation and just use it for transcoding existing video with handbrake or other CPU/memory intensive work where I don't need sound.

#11
sebus

sebus

    InsanelyMac Sage

  • Members
  • PipPipPipPipPip
  • 261 posts
  • Gender:Male
  • Location:UK
At this point one could get Gigabyte motherboard that natively does ML (maybe not dual CPU & only 16Gb or RAM, but zero other problems)

sebus

#12
wxe911

wxe911

    InsanelyMac Protégé

  • Members
  • Pip
  • 17 posts
Thank you sgreszcz for your explanation. You have invested a lot of time, not to mention money, into this sound nightmare.
Please report back again.

@ sebus -- yes one could indeed build as suggested. In this thread it is all about vitualisation. Thanks anyway.

Cheers

#13
wxe911

wxe911

    InsanelyMac Protégé

  • Members
  • Pip
  • 17 posts
My experience:
ESXi 5.1 on Gigabyte Z68MA-D2H-B3 current EFI Firmware. 32GB RAM, 2TB HDD. Boot ESXi of USB stick.

working:
Passthrough Onboard Realtek ALC889 sound chip.

Windows 7:
1) Sound working (headphone plugged into rear green 3.5mm jack) displayed through console.
2) Sound working (speaker attached to Windows 7 workstation) displayed through Remote Desktop Connection.
Note: 2) also works if 1) no passthrough is done in the VM.

Setup for Windows was painless and it seems to work quite well.


Mac OS X Mountain Lion 10.8.2:
Mountain Lion does not see any device.

Read a lot here:
#####
#####

Downladed (MultiBeast5.2.1) and (Kex_Beast). Made a copy of AppleHDA kext. Installed Alc889 driver using (MultiBeast5.2.1). Then extracted the AppleHDA and HDAEnabler kext from S/L/E folder onto desktop.
Used (Kext_Beast) to install them again, just to be safe that permissions are done correctly.

Now Mountain Lion does recognise the onboard alc889 sound chip as Built-in device and HD audio from the HD4000 graphics inbuilt i2600 CPU.

It does produce sound coming out from the rear green jack, plugged into ESXi host, but the volume is faint and a lot of crackling noise. Effectively not usable.
Was unable to find a solution to make it sound better.


Installed the iRAPP server, but was not able to get any sound out of it when connected with the iRAPP Windows client. No solution found at all.


Just wanted to share my experience with you.

Update:
Tried (CoRD_0.5.7), free opensource RDC client for Mac. Can connect to iRAPP server installed on Mountain Lion, but no sound.

Connect with Windows 7 RDC to iRAPP server and finally got sound good sound.

Edited by ErmaC, 08 April 2013 - 01:18 PM.
Still by-pass the forum word filter and ignore the forum rules.


#14
sgreszcz

sgreszcz

    InsanelyMac Protégé

  • Members
  • Pip
  • 13 posts

Just wanted to share my experience with you.


Thanks for the input, I appreciate it. I have never messed much with kext files or anything, as the only hackintosh I have running is on an old Dell Mini9 netbook.

So here is some more of what I've tried:

I bought the Maya XTE PCIe audio card, but I could not get it to pass through to Mountain Lion and it caused me to lose connection from vSphere client and lock up the whole system forcing me to hard reboot. I tested this to see if it was only a problem with OSX, but it also caused the host to lock up when I tried passing the soundcard through to a Windows XP installation. I think it might be that the card is really a PCI card with a PCI/PCIe bridge and ESXi doesn't like that. I am returning the card...

I experimented with a couple of "virtual soundcards" for OSX, but couldn't really get them to work or they weren't supported in a virutal environment. I installed the latest soundflower virtual sound driver, and I did get really good sound when I tested capturing a youtube video using Snapz Pro X. This will work OK for that sort of thing. I can probably work remotely with splashtop2 which does a good job at remote streaming of video and audio. I haven't tried it from the virtualized Mountain Lion install yet.

I forgot to mention that I did manage to pass through the ASUS Mio 888 sound card, but it wasn't detected natively in Mountain Lion and I haven't dug much into how to get it working in ML with kexts (which I'm trying to avoid). I have no idea how to get the AppleHDA.kext to work with this soundcard. Maybe I'll try again...

I also tested a plantronics USB soundcard virtual device by passing it through to ML. It was detected natively by OSX and I got sound, however it was choppy and crackly. Now that I've managed to get ML running on ESXi5.0 u2 (which allows USB device passthrough - ESXi 5.1 does not) I will try again some time with one of the passed-through USB controllers from my motherboard and see if the performance is better.

I have also managed to run this server headless with graphics acceleration on by inserting a 100 ohm resistor into the analogue video port. This forces OSX to load the accelerated drivers without a monitor connected, otherwise you won't get the benefit of the GPU.

#15
sebus

sebus

    InsanelyMac Sage

  • Members
  • PipPipPipPipPip
  • 261 posts
  • Gender:Male
  • Location:UK
How to make VGA dummy

#16
sgreszcz

sgreszcz

    InsanelyMac Protégé

  • Members
  • Pip
  • 13 posts

How to make VGA dummy


Subus, thanks - I did do that directly on the VGA port on my Radeon card. The Radeon card I have has three connectors - VGA, HDMI and DVI-D. I didn't realise that the DVI is "D" - digital only - so it doesn't have any analogue signalling, and you can't use a DVI-I to VGA converter and connect to that with the 3 resistors. I was able to get it working with the VGA port, but you need to have a monitor, or the resistors, in place on the VGA port from first cold bootup of the ESXi host. You will not get any signal on the VGA if you plug it in later.

When the Radeon is on my motherboard, it seems to disable the onboard VGA and uses the Radeon to show BIOS and ESXi boot up sequence (on VGA or DVI, haven't tried HDMI). Then part of the way through the bootup the ESXi screen freezes until you start up Mountain Lion guest on the vSphere client, and then the video passthrough takes over and you get the OSX desktop on your monitor.

The performance is much better when the GPU is enabled, especially when using screen sharing.

For now I have sound and accelerated video on a headless setup. I will try the USB passthrough for mouse/sound/keyboard and I re-purchased the ASUS MIO 888 soundcard to see if I can get it working in passthrough with kext modifications. This would give me a good workstation configuration.

#17
qball4

qball4

    InsanelyMac Protégé

  • Members
  • Pip
  • 2 posts
Any luck on the MIO 888 card? I am in the same boat with an ATI 6770 passed through perfectly but sound not working. I have tried (all with optical):

Maya U5 USB - on devices but no actual sound, works OOB on MBP 10.8.2
C-Media USB - on devices but no actual sound, works OOB on MBP 10.8.2
PCI Passthrough On-board ALC 889 - crackling/distortion, low volume

I thought perhaps the USB devices weren't getting enough bandwidth if sharing with all of the other devices I have plugged in so tried passing through pcie controllers:
Startech NEC 720200 USB 3.0 controller - esxi recognizes correctly, vm doesn't get past vmware screen, won't even go to bios setup, will however boot fine into Win7 vm... EDIT: Will boot to osx if no USB devices are plugged in, will not recognize any plugged in after boot
Frisco Logic 1009 USB 3.0 controller - esxi recognizes as "Unknown device" but will pass through, OSX does not recognize at all

#18
qball4

qball4

    InsanelyMac Protégé

  • Members
  • Pip
  • 2 posts
I found a solution in case anyone else is trying the same:

I used an NEC chipset (PD720100 or PD720101) based iogear GIC251U PCI card in passthru then connected the AudioTrak Maya U5 and voila! it works!

Final VM setup:
esxi 5.1 patched to 1021289 (patch necessary for usb controller passthru support)
OS X 10.8.3 using Donk's unlocker
XFX 6770 1GB (HD-677X-ZAF4) in passthru - flashed to 5770, full QE/CI no kexts needed
iogear GIC251U PCI usb 2.0 card in passthru - works OOB
AudioTrak Maya U5 usb sound card - optical out 5.1, works OOB

tip: remember to re-select the usb card for passthru under configuration/advanced settings after every hypervisor reboot as there is still a bug in esxi that will remove it after two reboots

If anyone finds a way to get usb 3.0 working under esxi/OSX please share!

#19
wxe911

wxe911

    InsanelyMac Protégé

  • Members
  • Pip
  • 17 posts
@sgreszcz

The ASUS MIO 888 sound card is based on the Realtek ALC892

You may want to try this kext http://www.osx86.net...&id=3336&page=1

An easy way to deal/install kexts is with (####) found here ####

As always backup your original kext before installing modded ones.

Edited by ErmaC, 08 April 2013 - 01:16 PM.
We have word filter and forum rules.


#20
sgreszcz

sgreszcz

    InsanelyMac Protégé

  • Members
  • Pip
  • 13 posts
Thanks for the tips. I haven't had the time to continue working on the sound and the MIO888. I'll report back when I get some more time to mess with this.





0 user(s) are reading this topic

0 members, 0 guests, 0 anonymous users

© 2014 InsanelyMac  |   News  |   Forum  |   Downloads  |   OSx86 Wiki  |   Mac Netbook  |   PHP hosting by CatN  |   Designed by Ed Gain  |   Logo by irfan  |   Privacy Policy