0. Preface
This is some preliminary work for coding a patch for Chameleon RC4/5, which will read the VBIOS of ATI (3*** HD & 4*** HD) and NVIDIA cards (nearly all models) and
modify existing VESA modes (such as 800x600) to the native resolution of the display.

For Widescreen monitors this will lead to an unstretched GUI & Spinner. If your extremely eager you can mod ur video rom right now and reflash it.; but im working on a patch similar to 915res to patch the bios on the fly...

1. Structure of VESA modes in ATI VBIOS

In ATI VBIOS'es the VESA modes are stored in a table 1064 bytes long, containing 38 Modes, each 28 bytes long.
The offset is not constant, but always around 0xA600. (There are actually two tables, but one I could identify as ComponentVideoInfo. Since this doesn't apply to us (& since i tried to patch this table w/o results) I'm concentrating on the table named above.

Here is a sample mode from an 4870HD:



Important for us are bytes 1, 2, 5, 6:

Bytes 1&2 (In this example 20 03) are byteflipped versions of the width in hex format. Eg: If you unflip them you get 0320 which is HEX for 800.
The same applies for bytes 5&6 ->58 02 -> Unflip them -> 0258 -> convert to DEC -> Height is 600.
This sample mode thus has the resolution 800x600.
The other bytes afterwards are for color depth, various sync times and pixel clocks

In every 3*** - 5*** HD bios there are five 800x600 modes after each other, each with different color depth, various sync times and pixel clocks.
For this fix to work we change those five modes to the resolution we want:

If native res of the display is 1680x1050, we convert those to hex, resulting in 0x0690 and 0x041A
Now we flip the bytes, resulting in 0x9006 and 0x1A04.

If we now replace the width & height in the afore mentioned five modes, we will get native 1680x1050x32, 1680x1050x16 & 1680x1050x8.

This is confirmed working on ati 3***, 4*** and 5***

I attached a small c++ tool to find the locations of those 5 modes.

2. Structure of VESA modes in NVIDIA VBIOS
In NVIDIA VBIOS'es the modes are stored in a similar byteflipped way:

Table 1
Update:
Thanks to bitrips report, i had another look at his bios & found another table in a different format.
(While ATI released some basic structures of their vbios and I could such identify the right table, NVIDIA doesn't)

This other table I found is 288bytes long, containing 32 modes á 9 bytes. The offset is again not stable.

Sample Code from 8800GT (XFX 512MB)


Resolution is byte flipped again, this time byte 1&2 and 3&4 (Sample: 320x200). The modes in this table go up to 1920x1200 (80 07 B0 04)



The other mode table is 512 bytes long, containing 16 modes, each 32 byte long.

Here is a sample mode from an 8800GTS



Important for us here are bytes 1, 2, 15, 16:

Bytes 1&2 (In this example 00 04) are byteflipped versions of the width in hex format. Eg: If you unflip them you get 0400 which is HEX for 1024.
The same applies for bytes 15&16 ->00 03 -> Unflip them -> 0300 -> convert to DEC -> Height is 768.
This sample mode thus has the resolution 1024x768.

Unlike the ATI modes Table, the NVIDIA is more compact, so we only need to modify two, sometimes only one mode.
So we convert the native resolution of the display as above and replace those values.

I attached a small c++ tool to find the locations of this mode.

3. Tool to Analyse VBIOS:
0. This tool only looks at where/how many VESA tables there are. It doesn't actually change them
1. Extract VBIOS using Windows etc.
2. run in Terminal ./analyseVBIOS bios.rom
3. You'll get the location of the tables
4. Zip contains exec and sources

My thank goes to:
The xorg ati team
This post for the original cl tool
The developers of 915res