Jump to content

Mavericks kernel testing on AMD (formerly Mountain Lion kernel testing on AMD)


theconnactic
 Share

6,414 posts in this topic

Recommended Posts

 

As for the arch i386 kernel, i don't think it's a good idea: the kernel extensions aren't open source, it isn't easy, not to say not possible at all, to make 32-bit version of all of them, or any of them, better saying.

 

On the latest iterations of OSX, we had the "easy way" (quoting David Elliot): stick to -legacy mode (pure 32-bit mode) or the way most of the patched kernels were: arch i386 kernels with -force64 (64-bit user land) boot flag. We can avoid it anymore: Mountain Lion is designed to be purely 64-bit, so both the easy way (that saved many non-ssse3 AMD CPU users with Lion, despite the severe limitations) and the 64-bit user land (which prevailed until Snow Leopard) won't help anymore: we must face the challenge of making a full 64-bit patched kernel, and maybe even a ssse3 emulator.

 

 

 

Hi, Zach.

 

IMO, you are completely right. I intend, as a first try, to do a rollback to the Lion version of com.apple.kec.corecrypto, if it exists.

 

 

 

Hi, zchef2k!

 

I don't know. What's the relationship did you perceive between this piece of code and a possible remaining cpuid.c issue?

 

What do i intend to do now:

 

1) Remove core crypto altogether, to repeat the hang zchef2k experienced. Then, do the rollback for the Lion version, if there is one;

 

2) Patch the bcopy.s (only the bcopy) of the Mountain Lion XNU using this diff file: xnu-1228.7.58_voodoo_release1.0.diff.zip

 

3) Combine the to options above.

 

I'll do it later tonight or, more probably, tomorrow morning, because there's an important download in progress in my Lion hackintosh right now, and it's a really large file, so it will probably take hours to finish. That's why i uploaded the diff: if someone doesn't want to wait to tomorrow, can do it right now (don't forget to keep a backup of the original bcopy.s). If somebody actually do it, please post a screenshot here. :)

 

 

 

Yes, i am, thanks to PookyMacMan, who kindly pointed me to the #lion and #MountainLion channels of the ircosx86.hu so i could get advice from meklort and nawcom about this little work of ours.

Yup, it's a bad idea. A terrible idea. But that's not to say it can't be done, even though I don't recommend it.

And about the extensions not being open source, well, we're breaking the EULA doing this, no? Just sayin.

Welcome to the party.

 

Everyone on this thread is on IRC, right?

I haven't been on IRC for a while, but I'll try to get on sometimes. Do we want our own #LegacyKernel channel? :3

Link to comment
Share on other sites

It would make sense to start our own channel as well, so that it's always on topic

 

I'll start the channel as soon as I get XChat installed. :)

 

Edit: Done. #LegacyKernel on irc.osx86.hu

Link to comment
Share on other sites

Alright, I'll join when I can! :)

 

Edit: No one is there...lol :P

 

Edit 2: Now it's comin' alive. :D

 

Some links for you:

 

http://code.google.c...E3 Emulator.pdf

 

http://code.google.c...SourceCode?tm=4

  • Like 1
Link to comment
Share on other sites

Yup, it's a bad idea. A terrible idea. But that's not to say it can't be done, even though I don't recommend it.

And about the extensions not being open source, well, we're breaking the EULA doing this, no? Just sayin.

 

Man, i'm not talking about this EULA thing. :)

 

The problem is that we don't have the sources for changing and recompiling the kexts. No source, no joy. We could try to reverse engineering them, but to reverse engineering just one of them, it could take months, assuming specialized people help us. To get things worse, we'd have to reverse-engineer dozens of them.

 

No, the idea isn't bad at all! But i don't think it's feasible.

Link to comment
Share on other sites

Man, i'm not talking about this EULA thing. :)

 

The problem is that we don't have the sources for changing and recompiling the kexts. No source, no joy. We could try to reverse engineering them, but to reverse engineering just one of them, it could take months, assuming specialized people help us. To get things worse, we'd have to reverse-engineer dozens of them.

Besides which, we would probably get in plenty of trouble for reverse-engineering (some of) the kexts - that and binary patching Apple isn't too happy with, so technically running an AMD hack is (kinda sorta) illegal...but not really. :P [explain]The EULA is not enforceable by law, the worst Apple can do is sue the offender, and that's only in the US, and they would only sue companies not individuals. On the other hand, reverse-engineering and binary patching of some things is a DMCA infringement, a much more serious offense. Again, only in the US[/explain]

Link to comment
Share on other sites

Besides which, we would probably get in plenty of trouble for reverse-engineering (some of) the kexts - that and binary patching Apple isn't too happy with, so technically running an AMD hack is (kinda sorta) illegal...but not really. :P [explain]The EULA is not enforceable by law, the worst Apple can do is sue the offender, and that's only in the US, and they would only sue companies not individuals. On the other hand, reverse-engineering and binary patching of some things is a DMCA infringement, a much more serious offense. Again, only in the US[/explain]

 

What if I do all the reverse engineering? Canadaaaaaaaaaaaaaaa ftw! :P

Just kidding. Yeah, I can see why it'd take so long, lol. Also, binary patching XNU isn't illegal at all - we just need to mention that the finished result is released under the APSL.

  • Like 1
Link to comment
Share on other sites

Patching XNU is not an issue (I even said so earlier myself) - binary patching some of the system daemons and files are. The ones that are not open source. I think libsystem.dyld and a couple others...? Can't remember off the top of my head since I never owned an AMD box, only legacy Intel...

Link to comment
Share on other sites

Patching XNU is not an issue (I even said so earlier myself) - binary patching some of the system daemons and files are. The ones that are not open source. I think libsystem.dyld and a couple others...? Can't remember off the top of my head since I never owned an AMD box, only legacy Intel...

Ohh I see. Stupid annoying American copryright laws! Up here we can decompile, reverse engineer, and patch whatever we want while sitting on a moose in an igloo watching hockey. :D

 

Anycase though, judging by the amount of people who use the "bay full of pirates" (A DMCA offence) on a daily basis I don't think we have much to worry about, even if we do a bit of reverse engineering. :)

Link to comment
Share on other sites

never saw about xnu-dev project. is the project that i whas talking about.

lets join it!

 

xnu-dev

A fork of the Apple® Darwin XNU kernel for generic x86 PCs

 

http://code.google.com/p/xnu-dev/

 

they describe a method to emulate a non supported instruction. (see "SSE3 Emulator.pdf")

the bottleneck is in the trap overhead, which i think that can be improved (actually adds about 430 cpu cycles). emulating instruction takes few cpu cycles.

Edited by bcobco
  • Like 1
Link to comment
Share on other sites

Hi, bcobco! Did you find the source code for their sse3 emulator inside that site? It isn't in the .pdf, which i'd already have. And yes: i think it's better to continue an existing project - and i think voodoo team won't mind, since they abandoned it - than start a new fork all from the beginning.

 

Important news: today we have a new soldier in our ranks. In fact, a veteran and decorated hero of the AMD kernel crusades. I'm talking about Nawcom, who is since this morning trying his patches on the Mountain Lion kernel. With his help, our chances are vastly improved: Nawcom, as anyone here should know, is the author of the very successful Snow Leopard legacy kernel. Regardless of his success at the short term, i think very highly valuable info and insights will come from his work.

 

As for me, i still haven't tried those experiences i talked about friday, but today is the day. I'll keep you updated.

  • Like 1
Link to comment
Share on other sites

Hi, bcobco! Did you find the source code for their sse3 emulator inside that site? It isn't in the .pdf, which i'd already have. And yes: i think it's better to continue an existing project - and i think voodoo team won't mind, since they abandoned it - than start a new fork all from the beginning.

 

Important news: today we have a new soldier in our ranks. In fact, a veteran and decorated hero of the AMD kernel crusades. I'm talking about Nawcom, who is since this morning trying his patches on the Mountain Lion kernel. With his help, our chances are vastly improved: Nawcom, as anyone here should know, is the author of the very successful Snow Leopard legacy kernel. Regardless of his success at the short term, i think very highly valuable info and insights will come from his work.

 

As for me, i still haven't tried those experiences i talked about friday, but today is the day. I'll keep you updated.

 

Nawcom's back? I'll give him my best wishes for it's successfull work and wish him all luck he can get to get through his medical issues.

  • Like 1
Link to comment
Share on other sites

UPDATE: No luck yet: the voodoobuild.sh is not working. Without it, all i have is two diff files (one for bcopy_64_sse3.s, other for bcopy_64_ssse3.s) that are incomplete and so useless.

 

Trying to remove corecrypto, to see where it takes me (i think i know, but...)

Link to comment
Share on other sites

Hi, folks!

 

Just summarizing what we've already achieved, to help newcomers in this topic to get a grasp of what's going on:

 

- Firstly, we tried removing bad ID calls inside XNU/osfmk/i386/cpuid.c and a few other files, as stated in post #83. Limiting ourselves to do this, we could get the system to boot on an AMD machine to the point all hfs files are read, then it freezes in a black screen before the point console output would start (the "vanilla" behavior on an unsupported CPU would be an instant reboot);

 

- Realizing it would be not enough, i decided to apply the highly successful AMD patch from RAWx86 (bronzovkaANVoodoo) to the Mountain Lion XNU kernel. Since the patches were designed for use with 10.7.4 Lion, i had to do myself some adaptations to the kernel to compile. The output kernel is at the post #118. This kernel gives us a kernel panic backtraced to corecrypto.kext, as you can see in post #122, right after the kernel starts to run. Anyway, it takes us farther than our first experiment;

 

- Removing corecrypto.kext, or altering its info.plist (in my case, i change the last string line to Safe Boot) gets the system frozen after a few lines of console output, as you can see in post #132. That's the current situation for now.

 

Possible causes to investigate:

 

1) As stated by zachensoul, the issue could lie on a necessary answer corecrypto must give to the system, and it's not doing so, so while keep it returns us a kernel panic, remove it won't work - and in fact does not work - since the system needs that answer to be correct to running properly. The possible solution for it would be it has to be fooled into returning the answer the system is looking for. However, the obstacle is how to trick it: zchef2k tried some changes, to no avail, as stated in post #142;

 

2) Maybe this is an issue caused by the lack of some CPU instructions Intel processors from core 2 on have, like the much discussed suplemental SSE3. Well, i think now it's time to go deep in that ssse3 emulator thing, be this the case or not.

 

One more thing: we now have a chat room for AMD and other unsupported CPUs kernel discussion in IRC, in the server osx86.hu: #LegacyKernel. Join us!

 

For now, this is it.

Link to comment
Share on other sites

Hi, nice folks!

 

i've been reading pictures of our corecrypto-related kernel panics. I really really wanted to understand the panic debugging info, so i googled for advice and found this:

 

http://developer.app...063/_index.html

 

There i learned that the message at the end of the line one, "type 6 = invalid opcode", means a textual description of the error, the number "6" meaning the exception vector in decimal. As the guide recommended me to do, i look for it at the volume 3A of the intel CPU guide (Intel 64 and IA-32 Architectures Software Developer’s Manual, Volume 3A: System Programming Guide, Part 1) and found this:

 

 

Interrupt 6—Invalid Opcode Exception (#UD)

 

 

Exception Class Fault.

Description

Indicates that the processor did one of the following things:

  • Attempted to execute an invalid or reserved opcode.
  • Attempted to execute an instruction with an operand type that is invalid for its accompanying opcode; for example, the source operand for a LES instruction is not a memory location.
  • Attempted to execute an MMX or SSE/SSE2/SSE3 instruction on an Intel 64 or IA-32 processor that does not support the MMX technology or SSE/SSE2/SSE3/SSSE3 extensions, respectively. CPUID feature flags MMX (bit 23), SSE (bit 25), SSE2 (bit 26), SSE3 (ECX, bit 0), SSSE3 (ECX, bit 9) indicate support for these extensions.
  • Attempted to execute an MMX instruction or SSE/SSE2/SSE3/SSSE3 SIMD instruction (with the exception of the MOVNTI, PAUSE, PREFETCHh, SFENCE, LFENCE, MFENCE, CLFLUSH, MONITOR, and MWAIT instructions) when the EM flag in control register CR0 is set (1).
  • Attempted to execute an SSE/SE2/SSE3/SSSE3 instruction when the OSFXSR bit in control register CR4 is clear (0). Note this does not include the following SSE/SSE2/SSE3 instructions: MASKMOVQ, MOVNTQ, MOVNTI, PREFETCHh, SFENCE, LFENCE, MFENCE, and CLFLUSH; or the 64-bit versions of the PAVGB, PAVGW, PEXTRW, PINSRW, PMAXSW, PMAXUB, PMINSW, PMINUB, PMOVMSKB, PMULHUW, PSADBW, PSHUFW, PADDQ, PSUBQ, PALIGNR, PABSB, PABSD, PABSW, PHADDD, PHADDSW, PHADDW, PHSUBD, PHSUBSW, PHSUBW, PMADDUBSM, PMULHRSW, PSHUFB, PSIGNB, PSIGND, and PSIGNW.
  • Attempted to execute an SSE/SSE2/SSE3/SSSE3 instruction on an Intel 64 or IA-32 processor that caused a SIMD floating-point exception when the OSXMMEXCPT bit in control register CR4 is clear (0).
  • Executed a UD2 instruction. Note that even though it is the execution of the UD2 instruction that causes the invalid opcode exception, the saved instruction pointer will still points at the UD2 instruction.
  • Detected a LOCK prefix that precedes an instruction that may not be locked or one that may be locked but the destination operand is not a memory location.
  • Attempted to execute an LLDT, SLDT, LTR, STR, LSL, LAR, VERR, VERW, or ARPL instruction while in real- address or virtual-8086 mode.
  • Attempted to execute the RSM instruction when not in SMM mode.

 

 

Obviously, this by itself doesn't tell everything. Remember, could it be any of the cases above. But since our AMDs doesn't support SSSE3 instructions, this makes a strong case (no pun intended) for the ones i underlined above. Meklort told me to disassemble the kext and look for the invalid opcode, if it was the issue (now we know it was). He hinted to the terminal command otool -tVv, which i used to get an assembly listing for debugging info. Perhaps the panic tells us exactly for what to look for in the listing, so i'll keep on trying to decipher it.

 

P.S.: I'm using instant idiot's picture from post #143. Unfortunately i lost all the pics of my kp.

 

P.P.S.: Here's the output of the otool, just in case someone want to take a look (would help a lot) and don't want wasting time obtaining it himself: corecrypto debugging.txt

 

P.P.P.S.: The BSD process corresponding to the thread of the panic was not a kernel task. It's unknown. That should mean that it does not originate from within the kernel (which is good) but somewhere else. That seems to confirm that naughty corecrypto.kext is guilty.

Link to comment
Share on other sites

theconnactic please read carefully that "SSE3 Emulator.pdf" paper.

it explains what to do with processor interrupt number 6 (invalid op code).

i only readed once, but i remember that voodoo team played with that interrupt.

when int6 happens, they look to see if instruction is on unsupported, and act according that.

that people did a really good job. that paper offers a first solution and a second improved solution.

i think that the problem has more and better solutions, but it needs deep kernel and hardware knowledge.

the source code has dissapeared, but the algorithm is there. it just needs to be implemented.

Link to comment
Share on other sites

Ooh, I really wish I could be of any help. You guys are AWESOME. I mean it! Coming this far already :) Keep it up. Will be busy lately so I guess no time for beta test coming 4 weeks or so, but I'll stick to this topic and will follow it :D

 

You guys rock!

 

Greetings from Holland :)

Link to comment
Share on other sites

 Share

×
×
  • Create New...