The other day I remember running across mention of some way of explicitly turning on the dual-link dvi output of an Nvidia graphics card, but I can't find it now!
I have an ancient GeForce 8400 in a machine running Mavericks, and when I hook it up to a large monitor using a dual-link cable, and running at 2560 x 1440, the signal shows up at half resolution (i.e. doubled pixels). Using the same monitor and cable on a different machine and graphics card (running Snow Leopard) works correctly so I know the monitor and cable are fine.
It could be that I'm wasting my time with junky hardware and this card doesn't do dual-link, but it's what I have right now. One challenge with using a better graphics card (I have several) is that this computer, which in other ways is an excellent machine (Quad core Xeon), doesn't have the extra power connector needed by the higher end graphics cards so I can't even try the ones I do have.