2449 replies to this topic

[theconnactic]

Thank you, Pooky!

I'll follow your advice! In fact, i confess i could have resorted to it before, but i was wary of bothering them.



Update: already there, already being trolled, lol. But i'm failing to join the rooms you told me to, Pooky. I'm getting this annoying message:

You need to identify with network services to join the room "#lion" on "irc.osx86.hu".

Another update: joined them, finally. Never used IRC before, so i think i'm forgiven, lol!

Uptading again: talked with Nawcom. He was busy, neverthless he answered me. It was a good conversation, and he briefly explained  some of the issues he found when he tried to add support for AMD CPUs in Lion, and the real dimensions of the work we're trying to do. We didn't talked much about the ssse3 instructions and how to find them in the commpage though. Instead, we focused on the AMD patches themselves. It was very brief, as i said, and i expect we talk more about it. He is really a nice guy. He told me that he'll even try something with Mountain Lion on his Bulldozer machine. It will be great if he succeds.

Later, if i can, i'll try to talk with meklort (is his IRC nickname "meklort"?), if he goes online. Any news from this conversation, if it really happens, i'll post here. Thank you, Pooky, for the IRC chat tip.

[PookyMacMan]

You're welcome. In my spare time I'll see if I can have a word with one of them too.

[theconnactic]

Thank you again, Pooky!

I'm with the Mountain Lion XNU open at my screen at this very moment, lol. Two PDFs at its side: the Intel combo guide that bcobco discovered (http://www.intel.com...-2b-manual.html), and my Assembly-learn-it-yourself companion (http://www.drpaulcarter.com/pcasm/ - you dig it out, remember?). I'll rescan all the files in the commpage (http://www.opensourc...k/i386/commpage) to see if i find something out. If i manage to do it fast, i'll take a close look at the files at the x86_64 folder (http://www.opensourc...8/osfmk/x86_64/). I think the crucial files must be inside either one or the other.

Notice that the x86_64 doesn't contain the commpage, which is inside i386 instead. It's odd, since there's no 32 bit xnu-1699.26.8 in the real life...


EDIT: i'm also doing some useful extra reading, check this one out: http://www.macresear...or_extensions_0 - it's quite outdated (mentioning Tiger!), but gave some me some interesting insights.

[theconnactic]

Well, i talked to meklort. He was nice, too. Lot of patience with the loads of inept questions i just threw at him, lol.

I'm not going to explain what he told me exactly, because i need to understand it first. He told me, however, that someday he could well do something with AMD and Mountain Lion. But don't count on it for now, guys. That someday could well take three years, according to him.

By the way, i was right: 3) We know not.

It'll take a lot of effort, reading and learning for me to do the job. Alone, to be candid, it's probably above my capabilities for now. Long term goal, my friends. But i'm still on the front.

[bcobco]

im impressed. seems that this is a really hard task to fork xnu...

can you please tell us some conclusions of the interviews with those kernel hackers?

[theconnactic]

Hi, bcobco!

Essentially, they both (meklort and nawcom before him) said quite the same thing: it's an awfully tedious task, to find each and any incompatibility between the XNU and AMD CPUs and deal with it. Tedious and time-consuming. As for the ssse3 problem, maybe the instructions aren't really at the files we thought they were at first (this is one of the possibilities i raised before). Anyway, any crucial task performed using ssse3 must be dealth with, rewriting them so they could be performed using supported instructions. We didn't discuss anything about ssse3 emulation.

Anyway, it's very clear that is not an easy task for beginners (like me), since it requires advanced knowledge of C and assembly to perform and this knowledge won't be achieved overnight. The positive thing is that, according to meklort, it's possible to be done: there's no dead end here. He warn, though, about the bad quality of anterior patches, which have been applied without much criteria. He suggested that, if we are to do that despite of all difficulties, we try not to simply "hack it for AMD", but to alter the specific Intel-blocks to make the kernel more CPU-agnostic.

We talk about debugging too: as we should know, one of our greatest difficulties is to know what actually went wrong when something goes wrong. He told me about VMware (my first hackintosh experience was with a virtual machine; if i just knew... time for experimenting again), but VMware somehow blurs the whole thing. Other debugging possibilities are either using FireWire (that i confess i didn't understand) or adding the C function kprintf to the code. I'll try all but FireWire when it comes to the time.

We should (and i will, for sure) keep trying to do that, but it's not reasonable to expect short term results, unless some really capable person start working on it. Anyway, my road map now is to find these Intel specific pieces and start working on it. I'm reserving at least one entire hour a day to play with the kernel, since this is the maximum i can do regularly. Maybe we go slowly, but remember that, in the end, the tortoise defeated the hare.

[bcobco]

so it seems that would be nice to create an open project, an official fork of darwin/xnu (in github or similar), to make the kernel compatible with all x86 architecture, capable to handle incompatible instructions of userland code.


a huge challenge!

[theconnactic]

Precisely, bcobco! You speak my thoughts out!

Pursuing one of meklort's/nawcom's tips, i found this: http://www.hexblog.com/?p=94

Lucky me, i still have a Windows 7 64-bit installation in my desktop. Gotta brush the spider webs from it and put a little grease so it runs, lol. An eternity since i opened it last time.

EDIT: just to make myself clear, i'm'not looking forward to run OSX on a virtual machine. Virtualization is useful, but it hinder the OS capabilities, and i'd rather run Windows under OSX on a Parallels virtual machine (in fact, that's precisely what i do ).  What is implicated here is the use of VMware as a debugging tool.

[bcobco]

theconnactic, on 26 September 2012 - 11:42 PM, said:

Pursuing one of meklort's/nawcom's tips, i found this: http://www.hexblog.com/?p=94

huh, i took a look at that, but that knowledges escapes from my possibilities...

im happy with my objetive: since unfortunately linux is actually not suitable for audio production and windows does not convince me, i struggled hard to get an almost working osx system running with an overclocked custom build computer. its pretty stable, and gives very good audio performance for music production. thats why im very grateful with all the people that make this possible.

____
edit
it would be useful to collect all links and things collected in this thread, in the first topic. it would be useful for those who want to join this quest and find this thread,

Edited by bcobco, 27 September 2012 - 12:21 AM.

[theconnactic]

Take a look at this header: this is cpuid.h, included at the beginning of the commpage.c file. It seems quite obvious to me that it has a crucial role in making our XNU kernel intel-only. Notice the assignment of the much-dreaded ssse3 instructions to the HBit (9),  as well as the definitions of various intel CPUs.

Notice that it lacks Intel CPUs that are reportedly not capable to run Mountain Lion (perhaps are those definitions called in some kind of CPU check that prevents the kernel from running on unlisted CPUs, nut at the same time there's a definition for a "VMware CPU" and an "unknown CPU").

As a side note, the definition of Syscall and Sysret as external features: they're AMD children, but are universally supported as far as i know.

(occasional supplemental comments by me at the right side of lines marked with red underlined bold letters).

#ifndef _MACHINE_CPUID_H_
#define _MACHINE_CPUID_H_

#include <sys/appleapiopts.h>

#ifdef __APPLE_API_PRIVATE

#define CPUID_VID_INTEL "GenuineIntel"
#define CPUID_VID_AMD "AuthenticAMD" (theconnactic: look how funny! a definition for AMD CPUs! Why?)

#define CPUID_VMM_ID_VMWARE "VMwareVMware"

#define CPUID_STRING_UNKNOWN "Unknown CPU Typ"

#define _Bit(n) (1ULL << n)
#define _HBit(n) (1ULL << ((n)+32))

/*
* The CPUID_FEATURE_XXX values define 64-bit values
* returned in %ecx:%edx to a CPUID request with %eax of 1:
*/
#define CPUID_FEATURE_FPU _Bit(0) /* Floating point unit on-chip */
#define CPUID_FEATURE_VME _Bit(1) /* Virtual Mode Extension */
#define CPUID_FEATURE_DE _Bit(2) /* Debugging Extension */
#define CPUID_FEATURE_PSE _Bit(3) /* Page Size Extension */
#define CPUID_FEATURE_TSC _Bit(4) /* Time Stamp Counter */
#define CPUID_FEATURE_MSR _Bit(5) /* Model Specific Registers */
#define CPUID_FEATURE_PAE _Bit(6) /* Physical Address Extension */
#define CPUID_FEATURE_MCE _Bit(7) /* Machine Check Exception */
#define CPUID_FEATURE_CX8 _Bit(8) /* CMPXCHG8B */
#define CPUID_FEATURE_APIC _Bit(9) /* On-chip APIC */
#define CPUID_FEATURE_SEP _Bit(11) /* Fast System Call */
#define CPUID_FEATURE_MTRR _Bit(12) /* Memory Type Range Register */
#define CPUID_FEATURE_PGE _Bit(13) /* Page Global Enable */
#define CPUID_FEATURE_MCA _Bit(14) /* Machine Check Architecture */
#define CPUID_FEATURE_CMOV _Bit(15) /* Conditional Move Instruction */
#define CPUID_FEATURE_PAT _Bit(16) /* Page Attribute Table */
#define CPUID_FEATURE_PSE36 _Bit(17) /* 36-bit Page Size Extension */
#define CPUID_FEATURE_PSN _Bit(18) /* Processor Serial Number */
#define CPUID_FEATURE_CLFSH _Bit(19) /* CLFLUSH Instruction supported */
#define CPUID_FEATURE_DS _Bit(21) /* Debug Store */
#define CPUID_FEATURE_ACPI _Bit(22) /* Thermal monitor and Clock Ctrl */
#define CPUID_FEATURE_MMX _Bit(23) /* MMX supported */
#define CPUID_FEATURE_FXSR _Bit(24) /* Fast floating pt save/restore */
#define CPUID_FEATURE_SSE _Bit(25) /* Streaming SIMD extensions */
#define CPUID_FEATURE_SSE2 _Bit(26) /* Streaming SIMD extensions 2 */
#define CPUID_FEATURE_SS _Bit(27) /* Self-Snoop */
#define CPUID_FEATURE_HTT _Bit(28) /* Hyper-Threading Technology */
#define CPUID_FEATURE_TM _Bit(29) /* Thermal Monitor (TM1) */
#define CPUID_FEATURE_PBE _Bit(31) /* Pend Break Enable */

#define CPUID_FEATURE_SSE3 _HBit(0) /* Streaming SIMD extensions 3 */
#define CPUID_FEATURE_PCLMULQDQ _HBit(1) /* PCLMULQDQ instruction */
#define CPUID_FEATURE_DTES64 _HBit(2) /* 64-bit DS layout */
#define CPUID_FEATURE_MONITOR _HBit(3) /* Monitor/mwait */
#define CPUID_FEATURE_DSCPL _HBit(4) /* Debug Store CPL */
#define CPUID_FEATURE_VMX _HBit(5) /* VMX */
#define CPUID_FEATURE_SMX _HBit(6) /* SMX */
#define CPUID_FEATURE_EST _HBit(7) /* Enhanced SpeedsTep (GV3) */
#define CPUID_FEATURE_TM2 _HBit(8) /* Thermal Monitor 2 */
#define CPUID_FEATURE_SSSE3 _HBit(9) /* Supplemental SSE3 instructions */ (theconnactic: aha! here it is. Are this checked for in any routine? Probably an asm one. bcopy?)
#define CPUID_FEATURE_CID _HBit(10) /* L1 Context ID */
#define CPUID_FEATURE_SEGLIM64 _HBit(11) /* 64-bit segment limit checking */
#define CPUID_FEATURE_CX16 _HBit(13) /* CmpXchg16b instruction */
#define CPUID_FEATURE_xTPR _HBit(14) /* Send Task PRiority msgs */
#define CPUID_FEATURE_PDCM _HBit(15) /* Perf/Debug Capability MSR */

#define CPUID_FEATURE_PCID _HBit(17) /* ASID-PCID support */
#define CPUID_FEATURE_DCA _HBit(18) /* Direct Cache Access */
#define CPUID_FEATURE_SSE4_1 _HBit(19) /* Streaming SIMD extensions 4.1 */
#define CPUID_FEATURE_SSE4_2 _HBit(20) /* Streaming SIMD extensions 4.2 */
#define CPUID_FEATURE_xAPIC _HBit(21) /* Extended APIC Mode */
#define CPUID_FEATURE_MOVBE _HBit(22) /* MOVBE instruction */
#define CPUID_FEATURE_POPCNT _HBit(23) /* POPCNT instruction */
#define CPUID_FEATURE_TSCTMR _HBit(24) /* TSC deadline timer */
#define CPUID_FEATURE_AES _HBit(25) /* AES instructions */
#define CPUID_FEATURE_XSAVE _HBit(26) /* XSAVE instructions */
#define CPUID_FEATURE_OSXSAVE _HBit(27) /* XGETBV/XSETBV instructions */
#define CPUID_FEATURE_AVX1_0 _HBit(28) /* AVX 1.0 instructions */
#define CPUID_FEATURE_VMM _HBit(31) /* VMM (Hypervisor) present */
#define CPUID_FEATURE_SEGLIM64 _HBit(11) /* 64-bit segment limit checking */
#define CPUID_FEATURE_PCID _HBit(17) /* ASID-PCID support */
#define CPUID_FEATURE_TSCTMR _HBit(24) /* TSC deadline timer */
#define CPUID_FEATURE_AVX1_0 _HBit(28) /* AVX 1.0 instructions */
#define CPUID_FEATURE_F16C _HBit(29) /* Float16 convert instructions */
#define CPUID_FEATURE_RDRAND _HBit(30) /* RDRAND instruction */

/*
* Leaf 7, subleaf 0 additional features.
* Bits returned in %ebx to a CPUID request with {%eax,%ecx} of (0x7,0x0}:
*/
#define CPUID_LEAF7_FEATURE_RDWRFSGS _Bit(0) /* FS/GS base read/write */
#define CPUID_LEAF7_FEATURE_SMEP _Bit(7) /* Supervisor Mode Execute Protect */
#define CPUID_LEAF7_FEATURE_ENFSTRG _Bit(9) /* ENhanced Fast STRinG copy */

/*
* The CPUID_EXTFEATURE_XXX values define 64-bit values
* returned in %ecx:%edx to a CPUID request with %eax of 0x80000001:
*/
#define CPUID_EXTFEATURE_SYSCALL _Bit(11) /* SYSCALL/sysret */
#define CPUID_EXTFEATURE_XD _Bit(20) /* eXecute Disable */

#define CPUID_EXTFEATURE_1GBPAGE _Bit(26) /* 1GB pages */
#define CPUID_EXTFEATURE_RDTSCP _Bit(27) /* RDTSCP */
#define CPUID_EXTFEATURE_EM64T _Bit(29) /* Extended Mem 64 Technology */

#define CPUID_EXTFEATURE_LAHF _HBit(0) /* LAFH/SAHF instructions */

/*
* The CPUID_EXTFEATURE_XXX values define 64-bit values
* returned in %ecx:%edx to a CPUID request with %eax of 0x80000007:
*/
#define CPUID_EXTFEATURE_TSCI _Bit(8) /* TSC Invariant */

#define CPUID_CACHE_SIZE 16 /* Number of descriptor values */

#define CPUID_MWAIT_EXTENSION _Bit(0) /* enumeration of WMAIT extensions */
#define CPUID_MWAIT_BREAK _Bit(1) /* interrupts are break events */

#define CPUID_MODEL_YONAH 0x0E (theconnactic: here the CPU models. only C2duo and later ones! Are they called back in a CPU check routine?)
#define CPUID_MODEL_MEROM 0x0F
#define CPUID_MODEL_PENRYN 0x17
#define CPUID_MODEL_NEHALEM 0x1A
#define CPUID_MODEL_FIELDS 0x1E /* Lynnfield, Clarksfield, Jasper */
#define CPUID_MODEL_DALES 0x1F /* Havendale, Auburndale */
#define CPUID_MODEL_NEHALEM_EX 0x2E
#define CPUID_MODEL_DALES_32NM 0x25 /* Clarkdale, Arrandale */
#define CPUID_MODEL_WESTMERE 0x2C /* Gulftown, Westmere-EP, Westmere-WS */
#define CPUID_MODEL_WESTMERE_EX 0x2F
#define CPUID_MODEL_SANDYBRIDGE 0x2A
#define CPUID_MODEL_JAKETOWN 0x2D
#define CPUID_MODEL_IVYBRIDGE 0x3A


#define CPUID_VMM_FAMILY_UNKNOWN 0x0
#define CPUID_VMM_FAMILY_VMWARE 0x1

#ifndef ASSEMBLER
#include <stdint.h>
#include <mach/mach_types.h>
#include <kern/kern_types.h>
#include <mach/machine.h>


typedef enum { eax, ebx, ecx, edx } cpuid_register_t;
static inline void
cpuid(uint32_t *data)
{
asm("cpuid"
: "=a" (data[eax]),
"=b" (data[ebx]),
"=c" (data[ecx]),
"=d" (data[edx])
: "a" (data[eax]),
"b" (data[ebx]),
"c" (data[ecx]),
"d" (data[edx]));
}

static inline void
do_cpuid(uint32_t selector, uint32_t *data)
{
asm("cpuid"
: "=a" (data[0]),
"=b" (data[1]),
"=c" (data[2]),
"=d" (data[3])
: "a"(selector),
"b" (0),
"c" (0),
"d" (0));
}

/*
* Cache ID descriptor structure, used to parse CPUID leaf 2.
* Note: not used in kernel.
*/
typedef enum { Lnone, L1I, L1D, L2U, L3U, LCACHE_MAX } cache_type_t ;
typedef struct {
unsigned char value; /* Descriptor value */
cache_type_t type; /* Cache type */
unsigned int size; /* Cache size */
unsigned int linesize; /* Cache line size */
#ifdef KERNEL
const char *description; /* Cache description */
#endif /* KERNEL */
} cpuid_cache_desc_t;

#ifdef KERNEL
#define CACHE_DESC(value,type,size,linesize,text) \
{ value, type, size, linesize, text }
#else
#define CACHE_DESC(value,type,size,linesize,text) \
{ value, type, size, linesize }
#endif /* KERNEL */

/* Monitor/mwait Leaf: */
typedef struct {
uint32_t linesize_min;
uint32_t linesize_max;
uint32_t extensions;
uint32_t sub_Cstates;
} cpuid_mwait_leaf_t;

/* Thermal and Power Management Leaf: */
typedef struct {
boolean_t sensor;
boolean_t dynamic_acceleration;
boolean_t invariant_APIC_timer;
boolean_t core_power_limits;
boolean_t fine_grain_clock_mod;
boolean_t package_thermal_intr;
uint32_t thresholds;
boolean_t ACNT_MCNT;
boolean_t hardware_feedback;
boolean_t energy_policy;
} cpuid_thermal_leaf_t;


/* XSAVE Feature Leaf: */
typedef struct {
uint32_t extended_state[4]; /* eax .. edx */
} cpuid_xsave_leaf_t;


/* Architectural Performance Monitoring Leaf: */
typedef struct {
uint8_t version;
uint8_t number;
uint8_t width;
uint8_t events_number;
uint32_t events;
uint8_t fixed_number;
uint8_t fixed_width;
} cpuid_arch_perf_leaf_t;

/* Physical CPU info - this is exported out of the kernel (kexts), so be wary of changes */
typedef struct {
char cpuid_vendor[16];
char cpuid_brand_string[48];
const char *cpuid_model_string;

cpu_type_t cpuid_type; /* this is *not* a cpu_type_t in our <mach/machine.h> */
uint8_t cpuid_family;
uint8_t cpuid_model;
uint8_t cpuid_extmodel;
uint8_t cpuid_extfamily;
uint8_t cpuid_stepping;
uint64_t cpuid_features;
uint64_t cpuid_extfeatures;
uint32_t cpuid_signature;
uint8_t cpuid_brand;
uint8_t cpuid_processor_flag;

uint32_t cache_size[LCACHE_MAX];
uint32_t cache_linesize;

uint8_t cache_info[64]; /* list of cache descriptors */

uint32_t cpuid_cores_per_package;
uint32_t cpuid_logical_per_package;
uint32_t cache_sharing[LCACHE_MAX];
uint32_t cache_partitions[LCACHE_MAX];

cpu_type_t cpuid_cpu_type; /* <mach/machine.h> */
cpu_subtype_t cpuid_cpu_subtype; /* <mach/machine.h> */

/* Per-vendor info */
cpuid_mwait_leaf_t cpuid_mwait_leaf;
#define cpuid_mwait_linesize_max cpuid_mwait_leaf.linesize_max
#define cpuid_mwait_linesize_min cpuid_mwait_leaf.linesize_min
#define cpuid_mwait_extensions cpuid_mwait_leaf.extensions
#define cpuid_mwait_sub_Cstates cpuid_mwait_leaf.sub_Cstates
cpuid_thermal_leaf_t cpuid_thermal_leaf;
cpuid_arch_perf_leaf_t cpuid_arch_perf_leaf;
cpuid_xsave_leaf_t cpuid_xsave_leaf;

/* Cache details: */
uint32_t cpuid_cache_linesize;
uint32_t cpuid_cache_L2_associativity;
uint32_t cpuid_cache_size;

/* Virtual and physical address aize: */
uint32_t cpuid_address_bits_physical;
uint32_t cpuid_address_bits_virtual;

uint32_t cpuid_microcode_version;

/* Numbers of tlbs per processor [i|d, small|large, level0|level1] */
uint32_t cpuid_tlb[2][2][2];
#define TLB_INST 0
#define TLB_DATA 1
#define TLB_SMALL 0
#define TLB_LARGE 1
uint32_t cpuid_stlb;

uint32_t core_count;
uint32_t thread_count;

/* Max leaf ids available from CPUID */
uint32_t cpuid_max_basic;
uint32_t cpuid_max_ext;

/* Family-specific info links */
uint32_t cpuid_cpufamily;
cpuid_mwait_leaf_t *cpuid_mwait_leafp;
cpuid_thermal_leaf_t *cpuid_thermal_leafp;
cpuid_arch_perf_leaf_t *cpuid_arch_perf_leafp;
cpuid_xsave_leaf_t *cpuid_xsave_leafp;
uint32_t cpuid_leaf7_features;
} i386_cpu_info_t;

#ifdef MACH_KERNEL_PRIVATE
typedef struct {
char cpuid_vmm_vendor[16];
uint32_t cpuid_vmm_family;
uint32_t cpuid_vmm_bus_frequency;
uint32_t cpuid_vmm_tsc_frequency;
} i386_vmm_info_t;
#endif

#ifdef __cplusplus
extern "C" {
#endif

/*
* External declarations
*/
extern cpu_type_t cpuid_cputype(void);
extern cpu_subtype_t cpuid_cpusubtype(void);
extern void cpuid_cpu_display(const char *);
extern void cpuid_feature_display(const char *);
extern void cpuid_extfeature_display(const char *);
extern char * cpuid_get_feature_names(uint64_t, char *, unsigned);
extern char * cpuid_get_extfeature_names(uint64_t, char *, unsigned);
extern char * cpuid_get_leaf7_feature_names(uint64_t, char *, unsigned);

extern uint64_t cpuid_features(void);
extern uint64_t cpuid_extfeatures(void);
extern uint64_t cpuid_leaf7_features(void);
extern uint32_t cpuid_family(void);
extern uint32_t cpuid_cpufamily(void);

extern void cpuid_get_info(i386_cpu_info_t *info_p);
extern i386_cpu_info_t *cpuid_info(void);

extern void cpuid_set_info(void);

#ifdef MACH_KERNEL_PRIVATE
extern boolean_t cpuid_vmm_present(void);
extern i386_vmm_info_t *cpuid_vmm_info(void);
extern uint32_t cpuid_vmm_family(void);
#endif

#ifdef __cplusplus
}
#endif

#endif /* ASSEMBLER */

#endif /* __APPLE_API_PRIVATE */
#endif /* _MACHINE_CPUID_H_ */

Edited by PookyMacMan, 27 September 2012 - 04:25 AM.
Please use [code] [/code] tags around large portions of text like this, it makes navigating the post much easier. Thanks. Edit 2: Yikes, that didn't work well. Just remember the previous as a future principle.

[PookyMacMan]

Hm...could be something important indeed...

However, the next step would probably be to define what #define is (pardon the pun). From your knowledge of C, do you have any idea?

[theconnactic]

Hi, Pooky! Thanks for the answer.

Perhaps i can "define the define" (forgiven!): it's a C preprocessor directive that tells the preprocessor to replace with a specific value any ocurrence of a given character string. Its syntax is:

#define name_of_the_macro        body_of_the_macro

where the name of the macro is the character string and its body, the value specified.

For exemple, #define CPUID_FEATURE_SSSE3 _HBit(9) replaces the character string CPUID_FEATURE_SSSE3 for the value _HBit every time it appears in the file.

Or, at least, it's what i learned. There's people more skilled than i am that could wish to correct what i just wrote.

P.S.: I'll follow your advice about the code pastes, Pooky: thank you!

[bcobco]

in cpuid.c

/* this function is Intel-specific */
static void
cpuid_set_cache_info( i386_cpu_info_t * info_p )

/* verify we are running on a supported CPU */
if ((strncmp(CPUID_VID_INTEL, info_p->cpuid_vendor,
	 min(strlen(CPUID_STRING_UNKNOWN) + 1,
sizeof(info_p->cpuid_vendor)))) ||
(cpuid_set_cpufamily(info_p) == CPUFAMILY_UNKNOWN))
panic("Unsupported CPU");
info_p->cpuid_cpu_type = CPU_TYPE_X86;
info_p->cpuid_cpu_subtype = CPU_SUBTYPE_X86_ARCH1;

in fpu.c

/* Advertise SSE support */
if (cpuid_features() & CPUID_FEATURE_FXSR) {
fp_kind = FP_FXSR;
set_cr4(get_cr4() | CR4_OSFXS);
/* And allow SIMD exceptions if present */
if (cpuid_features() & CPUID_FEATURE_SSE) {
set_cr4(get_cr4() | CR4_OSXMM);
}
fp_register_state_size = sizeof(struct x86_fx_thread_state);
} else
panic("fpu is not FP_FXSR");

/*
* SSE arithmetic exception handling code.
* Basically the same as the x87 exception handler with a different subtype
*/
void
fpSSEexterrflt(void)

there are a lot of things to look at. i think that with proper knowledge, it should be possible to make darwin/xnu completely compatible with generic x86 and x86_64 architectures, not only intel specific hardware.



one legal question: can be forked the xnu source; in github, sourceforge, googlecode, or whatever public repository?
the idea is to create a darwin/xnu68 project. actually it seems that there is no project with this objetive (if there is some official project, lets join it.)

Edited by bcobco, 27 September 2012 - 09:05 AM.

[Lordadmiral Drake]

I did a cross-file-search for CPUID_FEATURE_SSSE3 and found it in the following locations:

..\osfmk\i386\cpuid.c line 888
..\osfmk\i386\cpuid.h line 97
..\osfmk\i386\machine_routines.c line 444

Can't say about the files in assembly code

In machine_routines.c

/*
  * Are we supporting MMX/SSE/SSE2/SSE3?
  * As distinct from whether the cpu has these capabilities.
  */
os_supports_sse = !!(get_cr4() & CR4_OSXMM);
if (ml_fpu_avx_enabled())
  cpu_infop->vector_unit = 9;
else if ((cpuid_features() & CPUID_FEATURE_SSE4_2) && os_supports_sse)
  cpu_infop->vector_unit = 8;
else if ((cpuid_features() & CPUID_FEATURE_SSE4_1) && os_supports_sse)
  cpu_infop->vector_unit = 7;
else if ((cpuid_features() & CPUID_FEATURE_SSSE3) && os_supports_sse)
  cpu_infop->vector_unit = 6;
else if ((cpuid_features() & CPUID_FEATURE_SSE3) && os_supports_sse)
  cpu_infop->vector_unit = 5;
else if ((cpuid_features() & CPUID_FEATURE_SSE2) && os_supports_sse)
  cpu_infop->vector_unit = 4;
else if ((cpuid_features() & CPUID_FEATURE_SSE) && os_supports_sse)
  cpu_infop->vector_unit = 3;
else if (cpuid_features() & CPUID_FEATURE_MMX)
  cpu_infop->vector_unit = 2;
else
  cpu_infop->vector_unit = 0;

And vector_unit is used here:

commpage.c

static void
commpage_init_cpu_capabilities( void )
{
uint32_t bits;
int cpus;
ml_cpu_info_t cpu_info;
bits = 0;
ml_cpu_get_info(&cpu_info);

switch (cpu_info.vector_unit) {
  case 9:
   bits |= kHasAVX1_0;
   /* fall thru */
  case 8:
   bits |= kHasSSE4_2;
   /* fall thru */
  case 7:
   bits |= kHasSSE4_1;
   /* fall thru */
  case 6:
   bits |= kHasSupplementalSSE3;
   /* fall thru */
  case 5:
   bits |= kHasSSE3;
   /* fall thru */
  case 4:
   bits |= kHasSSE2;
   /* fall thru */
  case 3:
   bits |= kHasSSE;
   /* fall thru */
  case 2:
   bits |= kHasMMX;
  default:
   break;
}
switch (cpu_info.cache_line_size) {
  case 128:
   bits |= kCache128;
   break;
  case 64:
   bits |= kCache64;
   break;
  case 32:
   bits |= kCache32;
   break;
  default:
   break;
}

kern_mib.c

case HW_VECTORUNIT: {
  int vector = cpu_info.vector_unit == 0? 0 : 1;
  return(SYSCTL_RETURN(req, vector));
}

And machine_routines.h

/* Struct for ml_cpu_get_info */
struct ml_cpu_info {
uint32_t vector_unit;
uint32_t cache_line_size;
uint32_t l1_icache_size;
uint32_t l1_dcache_size;
uint32_t l2_settings;
uint32_t l2_cache_size;
uint32_t l3_settings;
uint32_t l3_cache_size;
};

[bcobco]

how did you searched across files? thanks in advance
(i would like to search for "amd", "ssse3", "supplementalsse3", ... in all darwin source code)

seems that we -amd users- are in cpu_info.vector_unit: case 5: bits |= kHasSSE3;



http://darwinbuild.macosforge.org/
http://code.google.com/p/puredarwin/
i have found this project, but seems outdated (2009...)
if osx runs over darwin, can osx be run over puredarwin or whatever darwin fork ...?
it should be an hypothetical up to date version, that runs on generic x86 processo

Edited by bcobco, 27 September 2012 - 01:49 PM.

[Lordadmiral Drake]

In Win7 I went to kernel source code root folder and then entered Content: *text* in the search field. Don't know for OS X however

EDIT: Did a search for amd and found it in following files:

..\bsd\security\audit\audit_bsd.c
..\osfmk\i386cpuid.h
..\bsd\dev\i386\dis_tables.c
..\bsd\sys\dtrace.h
..\bsd\dev\i386\fasttrap_isa.c
..\bsd\dev\i386\fasttrap_regset.h
..\bsd\dev\i386\fbt_x86.c
..\bsd\kern\kern_fork.c
..\bsd\dev\memdev.c
..\bsd\nfs\nfs.h
..\bsd\sys\sdt_impl.h
..\osfmk\i386\trap.c

[theconnactic]

bcobco, on 27 September 2012 - 08:19 AM, said:

in cpuid.c

/* this function is Intel-specific */
static void
cpuid_set_cache_info( i386_cpu_info_t * info_p )

/* verify we are running on a supported CPU */
if ((strncmp(CPUID_VID_INTEL, info_p->cpuid_vendor,
	 min(strlen(CPUID_STRING_UNKNOWN) + 1,
sizeof(info_p->cpuid_vendor)))) ||
(cpuid_set_cpufamily(info_p) == CPUFAMILY_UNKNOWN))
panic("Unsupported CPU");
info_p->cpuid_cpu_type = CPU_TYPE_X86;
info_p->cpuid_cpu_subtype = CPU_SUBTYPE_X86_ARCH1;

in fpu.c

/* Advertise SSE support */
if (cpuid_features() & CPUID_FEATURE_FXSR) {
fp_kind = FP_FXSR;
set_cr4(get_cr4() | CR4_OSFXS);
/* And allow SIMD exceptions if present */
if (cpuid_features() & CPUID_FEATURE_SSE) {
set_cr4(get_cr4() | CR4_OSXMM);
}
fp_register_state_size = sizeof(struct x86_fx_thread_state);
} else
panic("fpu is not FP_FXSR");

/*
* SSE arithmetic exception handling code.
* Basically the same as the x87 exception handler with a different subtype
*/
void
fpSSEexterrflt(void)

there are a lot of things to look at. i think that with proper knowledge, it should be possible to make darwin/xnu completely compatible with generic x86 and x86_64 architectures, not only intel specific hardware.



one legal question: can be forked the xnu source; in github, sourceforge, googlecode, or whatever public repository?
the idea is to create a darwin/xnu68 project. actually it seems that there is no project with this objetive (if there is some official project, lets join it.)

Thank you very much, bcobco! I was intending to check cpuid.c last night, but i couldn't stand it - needed to get some sleep. Very useful info, and by putting together every piece, we're slowly solving this puzze.

As for you questions, 1) have in mind that i'm no lawyer, but i think that, being Darwin open source, there'd be no issues about forking it elsewhere and anyhow and 2) interesting idea of yours about that versions of Darwin. Your point is reasonable, but i think it would not be possible. See - Lion XNU is also Darwin: can we run Mountain Lion with it? No, we cannot. It's not just a question of having a CPU-agnostic version of Darwin; to run Mountain Lion, its associated specific version of Darwin should be either all-x86 compatible or hacked for AMD machines. But, well, we could compile it - the other Darwins - and try it just to see what happens.

LordAdmiralDrake, Well done! Very very valuable.

[bcobco]

theconnactic, on 27 September 2012 - 02:11 PM, said:

Thank you very much, bcobco! I was intending to check cpuid.c last night, but i couldn't stand it - needed to get some sleep. Very useful info, and by putting together every piece, we're slowly solving this puzze.

As for you questions, 1) have in mind that i'm no lawyer, but i think that, being Darwin open source, there'd be no issues about forking it elsewhere and anyhow and 2) interesting idea of yours about that versions of Darwin. Your point is reasonable, but i think it would not be possible. See - Lion XNU is also Darwin: can we run Mountain Lion with it? No, we can. It's not a question of having a CPU-agnostic version of Darwin; to run Mountain Lion, its associated specific version of Darwin should be either all-x86 or hacked for AMD machines. But, well, we could compile it - the other Darwins - and try it just to see what happens.

LordAdmiralDrake, Well done! Very very valuable.

yeah thats true, but i think thats because because osx-10.8 runs over darwin-12, and osx-10.7 runs on top of darwin-11. something similar would happen if you run osx-10.5 that runs over darwin-9, over another darwin version different from version 9.
not sure about this but seems logic. please correct me if im wrogn.

i mean, if you fork an arbitrary darwin version, the osx version must correspond to it.
http://en.wikipedia...._system)#Mascot

[theconnactic]

No, bcobco: you point seems quite reasonable to me. Really worth a try, why not? (remember that Puredarwin is a complete OS; what would be helpful for us, in theory, is its kernel)

EDIT: Puredarwin's kernel is out of question, at least for Mountain Lion. See why (http://www.puredarwi...pers/repository):



    ARCH

Target the architecture.

ARCH="i386"

At this time, only "i386" is supported, but we're convinced that minor efforts (e.g., adapting bootloader steps to different OpenFirmware, adding some needed projects specific/related to the PPC support in "pd_roots", etc...) should be involved in order to bring PPC support in Darwin 9. Idem for "X86_64".
Feel free to help us in this quest too.

[PookyMacMan]

theconnactic, on 27 September 2012 - 04:45 AM, said:

Hi, Pooky! Thanks for the answer.

Perhaps i can "define the define" (forgiven!): it's a C preprocessor directive that tells the preprocessor to replace with a specific value any ocurrence of a given character string. Its syntax is:

#define name_of_the_macro body_of_the_macro

where the name of the macro is the character string and its body, the value specified.

For exemple, #define CPUID_FEATURE_SSSE3 _HBit(9) replaces the character string CPUID_FEATURE_SSSE3 for the value _HBit every time it appears in the file.

Or, at least, it's what i learned. There's people more skilled than i am that could wish to correct what i just wrote.

I have a feeling that will be useful later on, thanks for that tip.

The PureDarwin issue: PureDarwin, though it does use a Darwin kernel, is more like Linux than Mac OS. The troubles I believe that AMD (and legacy Intel) people have are related to non-Darwin (a.k.a. OS X-specific) components. Although the changes needed for AMD/legacy Intel are in the kernel, the original problems occur (in my knowledge) because of interactions between the Mac OS components (kexts, system daemons etc.) and the kernel.

Also, something I'd like to know is whether or not getting legacy Intel to work would require the same work as AMD or more/less, or if it must be a totally different approach...I know that the main topic is AMD, but me and many others also have legacy Intel boxes.

Another thing I'm thinking is...let's stick with Lion XNU for now. ML has, from what I can see, a lot more architectural differences than Lion, so probably fixing Lion's issues first is the best idea.

Legality of forking XNU: totally legal. That's why we have patched kernels; an example of an actual code repository would be the old Voodoo kernel, produced back when dinosaurs of OSx86 roamed the earth (10.5)