So what I currently do is switch to the PCI sound card which temporarily solves the problem (but is of course a pain in the ass), and when the PCI sound card starts giving me crackly sound after a couple of hours I switch back to the USB sound card (again, a major pain in the ass).
Anyone knows how to diagnose and fix this?
EDIT: Found solution! - details here: http://forum.insanelymac.com/index.php?showtopic=112531