AppleIntelE1000e.kext for 10.8/10.7/10.6/10.5

[romaincs]

Hello,

 

I upgraded from 2.4.14 to this driver.

 

Console shows the "e1000_tx_map: failed to getphysicalsegment" message. I have not disabled TSO.

 

For now, it seems stable but my uptime is not really representative. I successfully trasfered about 24GB of data through afp, with a lot of "e1000_tx_map: failed to getphysicalsegment".

 

I'm still trying to get WoL working. I looked at the sources and answered my previous question (are diddl14 improvement are implemented). So, the answer is yes.

 

When I disable the kext, WoL works after shutdown. If not, magic packet won't power on the computer.

 

I'm looking for a way to disable this behavior inside the source, but I'm a bit "lost". I never did driver development but know some C/C++.

 

Romain

 

EDIT: It seems that "virtual IOReturn setWakeOnMagicPacket(bool active)" never get called. Shouldn't be called when ticking and unticking "wake from network access" from Energy Saving sys prefs ?

EDIT2: The only way to make Wake on Lan working is to disable the function "static void e1000_power_down_phy(struct e1000_adapter *adapter)". WoL works after this. (85267V-2, 0x808610ce). I guess it is a missing verification of wake on lan state ?

Edited July 14, 2014 by romaincs

[hnak]

You are right.

WOL code of Linux has changed since I investigated the part a long time ago.

 

I slightly modified shutdown code not to call e1000_power_down_phy() ( it used to be called in Linux at close(), but not now ) and found WOL works with GA-Z97X-UD7 TH (NIC = i217v) on Yosemite.

The source files are committed ( binary is not updated ), and you can build and try.

[pokenguyen]

Here is latest hnak's build with NETIF_F_TSO set to false. Cheers!

AppleIntelE1000e.kext.zip

[romaincs]

You are right.

WOL code of Linux has changed since I investigated the part a long time ago.

 

I slightly modified shutdown code not to call e1000_power_down_phy() ( it used to be called in Linux at close(), but not now ) and found WOL works with GA-Z97X-UD7 TH (NIC = i217v) on Yosemite.

The source files are committed ( binary is not updated ), and you can build and try.

Hey,

 

Thanks for the reply.

 

I tried it but it does not work. To make it work again, I have to comment the call in "int AppleIntelE1000e::__e1000_shutdown(bool runtime)".

 

I have only one issue with this modification :

If I put the computer in sleep it sleeps good. I can wake it with magic packet. but after a second sleep in a row, I got a partial sleep, some fans shutdown but most of hardware is still running.

 

Console shows "kernel[0]: Wake reason: ? (Network)" when I wake it first with magic packet, and then I get "kernel[0]: Wake reason: GBE (Network)" for the partial sleep (I did not send magic packet this time).

 

May be call "e1000e_reset" somewhere as the card seems to throw buggy wake events... ? I have to look at what is called at wake to put a reset and try...

 

Got another message in the console few min after this message (same state, no reboot): "kernel[0]: process WindowServer[188] caught causing excessive wakeups. Observed wakeups rate (per sec): 162; Maximum permitted wakeups rate (per sec): 150; Observation period: 300 seconds; Task lifetime number of wakeups: 83515".

 

It looks like NIC is flooding and I get strange wake up events.

 

[hnak]

Is WOL enabled in BIOS ?

 

Seemingly, the key is that when __e1000_shutdown is called, adapter->wol has non-Zero value and adapter->flags2 has FLAG2_HAS_PHY_WAKEUP set.

[romaincs]

Is WOL enabled in BIOS ?

 

Seemingly, the key is that when __e1000_shutdown is called, adapter->wol has non-Zero value and adapter->flags2 has FLAG2_HAS_PHY_WAKEUP set.

 

Yes, WoL is enabled.

 

For the windows driver to enable WoL after a shutdown (from windows), we have to set some settings in device. If you tick "Wake on network access" on mac os, nothing seems to happend in the driver.

 

should it be passed another way than "IOReturn AppleIntelE1000e::setWakeOnMagicPacket(bool active)" to the driver ?

[Mieze]

Yes, WoL is enabled.

 

For the windows driver to enable WoL after a shutdown (from windows), we have to set some settings in device. If you tick "Wake on network access" on mac os, nothing seems to happend in the driver.

 

should it be passed another way than "IOReturn AppleIntelE1000e::setWakeOnMagicPacket(bool active)" to the driver ?

 

What are you talking about? WoL from S3 or S5 because WoL from S5 isn't supported by OS X.

 

Mieze

[romaincs]

If S5 mean Shutdown state, WoL works with that edit. And S3 sleep state, then it works too.

[hnak]

Yes, WoL is enabled.

 

For the windows driver to enable WoL after a shutdown (from windows), we have to set some settings in device. If you tick "Wake on network access" on mac os, nothing seems to happend in the driver.

 

should it be passed another way than "IOReturn AppleIntelE1000e::setWakeOnMagicPacket(bool active)" to the driver ?

You can try by modifying code to set adapter->wol = 1 before calling __e1000_shutdown().

I found the WOL works with i217v, but not with 82579.  I found "phy wakup / mac wakeup" comment by the Linux driver author in  __e1000_shutdown, which may be related to the behavior.

[Mieze]

You can try by modifying code to set adapter->wol = 1 before calling __e1000_shutdown().

I found the WOL works with i217v, but not with 82579.  I found "phy wakup / mac wakeup" comment by the Linux driver author in  __e1000_shutdown, which may be related to the behavior.

 

As OS X doesn't support WoL from S5 officially, it doesn't call setWakeOnMagicPacket() on shutdown so that there is no clean solution. You have to resort to "some dirty trick" in order to get it working, i.e. enable WoL in systemWillShutdown().

 

Mieze

[hnak]

I have 3 MBs with onboard Intel NIC.

 

The following MBs wake up from power-off (even with the previous driver) :

Gigabyte GA-Z97X-UD7 TH (i217)

Asrock H97M Pro4 (i218v2)

 

The following MB does not wake up:

Intel DH68DB (82579)

 

It seems whether WOL works or not depends on BIOS/UEFI/Hardware.

I have not investigated the differences of the settings among the MBs, but WOL is enabled.

[romaincs]

You can try by modifying code to set adapter->wol = 1 before calling __e1000_shutdown().

I found the WOL works with i217v, but not with 82579.  I found "phy wakup / mac wakeup" comment by the Linux driver author in  __e1000_shutdown, which may be related to the behavior.

 

I will investigate that.

 

As OS X doesn't support WoL from S5 officially, it doesn't call setWakeOnMagicPacket() on shutdown so that there is no clean solution. You have to resort to "some dirty trick" in order to get it working, i.e. enable WoL in systemWillShutdown().

 

Mieze

 

It's not realy an OS issue, I think. To me, it's more like put the NIC in the previous state that bios setup before the driver drives it.

 

Without the driver, WoL works. So, the driver is setting the NIC in the "wrong" state as its (driver's) "default" behavior is to power off NIC on shutdown/sleep. A "backup/restore" state should do the trick, so, it could be clean, right ?

[Mieze]

It's not realy an OS issue, I think. To me, it's more like put the NIC in the previous state that bios setup before the driver drives it.

 

Without the driver, WoL works. So, the driver is setting the NIC in the "wrong" state as its (driver's) "default" behavior is to power off NIC on shutdown/sleep. A "backup/restore" state should do the trick, so, it could be clean, right ?

 

It is an OS X issue because of the lack of an interface how to enable WoL from S5.

 

Let's take a look on the sequence when the machine is going to S3 and WoL is enabled in System Preferences:

1. The driver's setWakeOnMagicPacket() method gets called.
2. The driver's disable() method gets called.
3. The driver's setPowerState() method gets called.

When the system shuts down you only get one call from the OS:

1. systemWillShutdown() where you have to take any actions required to stop the driver and put the NIC into the correct state.

The problem is that the driver doesn't get any information if it should enable WoL or not when the system is going down. You'll have to disabled it, as specified by the OS X, or enabled it regardless of the setting.

 

Mieze

[romaincs]

It is an OS X issue because of the lack of an interface how to enable WoL from S5.

 

Let's take a look on the sequence when the machine is going to S3 and WoL is enabled in System Preferences:

 

• The driver's setWakeOnMagicPacket() method gets called.
• The driver's disable() method gets called.
• The driver's setPowerState() method gets called.

When the system shuts down you only get one call from the OS:

• systemWillShutdown() where you have to take any actions required to stop the driver and put the NIC into the correct state.

The problem is that the driver doesn't get any information if it should enable WoL or not when the system is going down. You'll have to disabled it, as specified by the OS X, or enabled it regardless of the setting.

 

Mieze

Thanks for the information, now I have a better understanding of the OS side issue.

Maybe we could read at least the BIOS setting in the firmware ?

[hnak]

Thanks for the information, now I have a better understanding of the OS side issue.

Maybe we could read at least the BIOS setting in the firmware ?

It is already acquired in the line:

adapter->eeprom_wol |= E1000_WUFC_MAG;

[marmotta]

With Intel 82571EB dual nic and LACP i have very slow performance.... the only good kext is 2.4.14

[kode54]

I was running with version 3.0.4, which I just swapped out with a kextunload, copied in the latest, and kextload-ed it.

I'm awaiting results on this version, but with 3.0.4, when I attempted to run a Time Machine backup to my Time Capsule, or attempted to copy previous backups over my (unfortunately) 100Mbps link (spliced cable for reasons), it will eventually lose connection after about 13GB of transfer, due to the TX unit hardware hang.

This may be relevant, including the fix, which is probably impossible to do on OS X: http://serverfault.com/questions/193114/linux-e1000e-intel-networking-driver-problems-galore-where-do-i-start

Based on some feedback from a friend regarding Linux kernel drivers, how do I disable all of the "smart" features, like all the hardware offload things that are normally disabled by default in Windows? EDIT: Like Large Send Offload, which is buggy. (Ah, would that be TSO?)

 

EDIT 2: It appeared again with the latest version. Yes, it does seem I can reset the device and try again by kextunload/kextload cycling it. And I disabled TSO this time, we'll see if it fixes my issue.

 

Issue is specifically trying to copy 29 old local Time Machine backups to a newly purchased Time Capsule, totaling 130GB or 330k files, most of them mirrored throughout the set by the hard links that Finder is optimizing back into hardlinks on the target when it performs the copy.

 

Somebody remind me again why TSO is enabled by default? Because the Linux driver does it? Note how the Windows driver disables it by default, and often times, it actually slows down the connection throughput.

[hnak]

 

Somebody remind me again why TSO is enabled by default? Because the Linux driver does it? Note how the Windows driver disables it by default, and often times, it actually slows down the connection throughput.

No other reason.

I will change the default to false in future releases.

[kode54]

No other reason.

I will change the default to false in future releases.

 

Thank you.

 

I just thought I'd like to point out, that this feature is not broken through any fault of your own. It is likely broken in the hardware implementation, and that is why it is off by default. The fact that it can still be turned on in Windows is probably so know-it-alls can try to turn it back on, only to eventually discover why it was turned off in the first place.

 

Linux team was probably independent of the Windows team, so was not aware of the design error, or hadn't dealt with it enough.

 

I have an experienced and apparently wise Windows and Unix developer friend who even asked me about fragmentation offload when I told him about problems with my Intel NIC.

 

Then one of our other friends tried turning it on with his Windows machine, and found his gigabit line throughput decreased by 9% when he did so, and turned it back off again.

 

Incidentally, turning that off has fixed my lengthy copy operation, and I have not had any further TX unit failures since. At least experienced users may be interested to know that such a string of failures is soft recoverable, as the hardware appears to reset itself if the kext is unloaded and reloaded. Handy way to test out whether that TSO flag has any effect on things, at least.

[Mieze]

I just thought I'd like to point out, that this feature is not broken through any fault of your own. It is likely broken in the hardware implementation, and that is why it is off by default. The fact that it can still be turned on in Windows is probably so know-it-alls can try to turn it back on, only to eventually discover why it was turned off in the first place.

 

Linux team was probably independent of the Windows team, so was not aware of the design error, or hadn't dealt with it enough.

 

I have an experienced and apparently wise Windows and Unix developer friend who even asked me about fragmentation offload when I told him about problems with my Intel NIC.

 

Then one of our other friends tried turning it on with his Windows machine, and found his gigabit line throughput decreased by 9% when he did so, and turned it back off again.

 

Incidentally, turning that off has fixed my lengthy copy operation, and I have not had any further TX unit failures since. At least experienced users may be interested to know that such a string of failures is soft recoverable, as the hardware appears to reset itself if the kext is unloaded and reloaded. Handy way to test out whether that TSO flag has any effect on things, at least.

 

I'm sorry, but most of what you said is completely wrong:

• Intel's Windows driver suite enables all the offload features by default, at least for recent NICs.
• If TSO is broken it can be easily fixed with a new firmware image and this is the way it is done in the Windows world, but as parts of the Linux community reject proprietary firmware that is the point where the trouble starts.
• Linux and OS X are different with regard to the network system making it hard to port a driver. Most of the code needs to be rewritten in order to get a stable driver. Warping some kind of compatibility layer around the linux driver is definitely the wrong approach and almost inevitably results in a broken driver. I checked the driver's code and I would be very surprised if it works properly the way it was designed. Much of the code, in particular outputPacket() needs a complete rewrite in order to get rid of the linux overhead.
• TSO was designed to lower CPU usage, not to speed up network operation because any recent CPU is able to handle a gigabit connection at full speed without problems.
• It is true that TSO might cause serious speed issues because it has an impact on timing, in particular on the TCP data flow. That's the reason why you experience a speedup sometimes after TSO has been disabled.

Mieze

[hnak]

 Warping some kind of compatibility layer around the linux driver is definitely the wrong approach and almost inevitably results in a broken driver. I checked the driver's code and I would be very surprised if it works properly the way it was designed. Much of the code, in particular outputPacket() needs a complete rewrite in order to get rid of the linux overhead.

The decision was made to keep up with Linux update pace with minimal effort in the first place - I wanted to use Linux code without modification.

About outputPacket(), what kind of Linux overhead would you suggest to remove ?

[Mieze]

The decision was made to keep up with Linux update pace with minimal effort in the first place - I wanted to use Linux code without modification.

About outputPacket(), what kind of Linux overhead would you suggest to remove ?

Usually it's best to rewrite all the code involved with descriptor handling, starting from allocation, outputPacket(), rx/tx interrupt handling up to the cleanup routines when the driver gets disabled. This also allows to use IOBasicOutputQueue for concurrent tx/rx operations as both can be synchronized by starting/stoping the output queue.

 

Of course this makes updating the linux code basis much harder and would be a lot of work.

 

With regard to offload features (checksum calculation and TSO), it helps to read the Microsoft specification as OS X seems to conform widely to it. Besides that, I found out some facts which make it easier to implement the code:

• OS X doesn't offload packets with IPv4 header options/IPv6 extension headers for checksumming and TSO, so that the level 4 protocol header offset is always fixed, but there might be TCP options!
• For TSO there is no need to calculate the pseudo header checksum yourself because the network stack already placed it in the TCP checksum field of the TCP header when it offloads a packet. The IP header's checksum filed is zeroed too as stipulated by the MS specs.
• Most NICs, including those from Intel, require the complete headers (L2, L3 and L4) to be found in the first buffer for TSO so that a mbuf_pullup()-operation is usually required.

Mieze

[hnak]

Usually it's best to rewrite all the code involved with descriptor handling, starting from allocation, outputPacket(), rx/tx interrupt handling up to the cleanup routines when the driver gets disabled. This also allows to use IOBasicOutputQueue for concurrent tx/rx operations as both can be synchronized by starting/stoping the output queue.

 

Of course this makes updating the linux code basis much harder and would be a lot of work.

 

With regard to offload features (checksum calculation and TSO), it helps to read the Microsoft specification as OS X seems to conform widely to it. Besides that, I found out some facts which make it easier to implement the code:

• OS X doesn't offload packets with IPv4 header options/IPv6 extension headers for checksumming and TSO, so that the level 4 protocol header offset is always fixed, but there might be TCP options!
• For TSO there is no need to calculate the pseudo header checksum yourself because the network stack already placed it in the TCP checksum field of the TCP header when it offloads a packet. The IP header's checksum filed is zeroed too as stipulated by the MS specs.
• Most NICs, including those from Intel, require the complete headers (L2, L3 and L4) to be found in the first buffer for TSO so that a mbuf_pullup()-operation is usually required.

Currently, the driver copies the mbuf content to a continuous memory preallocated at startup and discards it immediately in OutputQueue(). Allocation does not happen during sending packets and pullup is not necessary.

The copy is a synchronous operation and may not be efficient as regard to CPU usage.

 

By the way, what is the benefit of IOBasicOutputQueue? Is concurrent rx/tx beneficial ?

[Mieze]

Currently, the driver copies the mbuf content to a continuous memory preallocated at startup and discards it immediately in OutputQueue(). Allocation does not happen during sending packets and pullup is not necessary.

The copy is a synchronous operation and may not be efficient as regard to CPU usage.

 

By the way, what is the benefit of IOBasicOutputQueue? Is concurrent rx/tx beneficial ?

 

Instead of copying the mbuf content it is more efficient to use an IOMbufNaturalMemoryCursor in order to create a scatter-gather-list that can be used directly to fill out the descriptors. The mbuf needs  to be attached to the descriptors in any way so that it can be freed by the tx interrupt handler when the request is complete. For an example please see my latest Project, the Atheros driver http://www.insanelymac.com/forum/topic/300056-solution-for-qca-ar816x-ar817x-and-killer-e220x/. It is not as complex as the Realtek driver but implements the same mechanisms.

 

IOBasicOutputQueue doesn't synchronize with the driver's work loop so that outputPacket() will continue to be called while the driver is handling interrupts, e.g. is receiving packets. Besides the speedup of operation you also get rid of the CPU overhead caused by synchronization.

 

Mieze

[hnak]

Instead of copying the mbuf content it is more efficient to use an IOMbufNaturalMemoryCursor in order to create a scatter-gather-list that can be used directly to fill out the descriptors. The mbuf needs to be attached to the descriptors in any way so that it can be freed by the tx interrupt handler when the request is complete. For an example please see my latest Project, the Atheros driver http://www.insanelymac.com/forum/topic/300056-solution-for-qca-ar816x-ar817x-and-killer-e220x/. It is not as complex as the Realtek driver but implements the same mechanisms.

 

IOBasicOutputQueue doesn't synchronize with the driver's work loop so that outputPacket() will continue to be called while the driver is handling interrupts, e.g. is receiving packets. Besides the speedup of operation you also get rid of the CPU overhead caused by synchronization.

 

Mieze

The code to copy mbuf is originally for jumbo frame and scatter-gather code exists for long ( I recently disabled it by a macro for simplicity ). IOBasicOutputQueue sounds easy to use. I will check it.