Yes, boot the mod cd, then at chameleon (the GUI boot loader) swap it for the OS X retail disk, press F5 a few times to update the display (your OS X disk will now appear), select the OS X disk using the arrow keys, then type -v (this will boot in verbose mode so we can monitor the boot process).
When you get to the installer, format your hard drive using disk utility (in the utilities menu) as GUID partition table (from the advanced menu), and OS X Extended Journaled.
Quit disk utility, the installer should start again.
Follow the installer through, (when installing, keep moving the mouse every ~10 mins to keep the screen from sleeping)
When the installer finishes it will say 'install failed' this is because it cannot install the boot loader to the EFI chip (you don't have one on your motherboard).
Reboot, and boot using the Mod CD, at chameleon, select your new OS X partition on your hard drive, type -v and press enter.
If you get to your desktop, great, you can now install chameleon to your hard drive so you no longer need the Mod CD and complete your install by finding kexts for your natively un-supported hardware.
You will place these in either System/Library/Extensions or Extra/Extensions of your hard drive (depening on the kext and the OS, e.g. Lion needs all kexts in S/L/E.)
Rebuild caches using kext utility before you reboot !!!IMPORTANT!!!
Reboot and pray
Always boot with -v until you have a stable install that boots every time.
If at any point you get stuck, take a photo of the panic screen, post it here and if no-one gets back to you, send me a pm with a link to the thread and I'll try to help you.