Linux and Windows UEFI boot using Tianocore DUET firmware

[srs5694]

Actually I made those changes locally in my system and pushed it only yesterday.

 

I've uploaded a new version and updated the link above; but to repeat it so you don't need to find it, it's:

 

http://www.rodsbooks.com/duet-install-0.1.1.tgz

 

Regarding your installation script, I've modified mine to use it, but I've discovered one issue: Because you use the sudo command in the script, it won't work when run from PartedMagic, which doesn't include sudo. It'd be nice to be able to run it from PartedMagic, so as to prepare a hard disk before installing an OS on it.

 

The 2.3 won't stay 2.3 forever. But its your call. AFAIK there is no noticable difference between the 2.1 and 2.3 firmwares except when AHCI and USB keyboards are involved.

 

For me, 2.3 doesn't work at all on one of my computers, but 2.1 does. (I wish it were the other way around, obviously!) 2.3 crashes with a page fault or illegal instruction or some such message.

 

The e2fsprogs blkid has been deprecated by e2fsprogs devs a long time back. Although I use Archlinux which is rolling release with always the recent version in the repos, I don't think even Debian users will need to bother about blkid.

 

The new script checks the blkid version and uses it if it's the util-linux version, but keeps the old code in case it's not.

 

FWIW, I've also changed the way it splits off the whole-disk device filename and partition number. This has enabled me to eliminate all use of udevadm, which should make the script more portable; but there's the drawback that the new method is built on assumptions about how the device nodes are named (where the split between the whole-disk device and partition number occurs, mainly). This should be OK on hard disks using conventional names, but not if somebody uses a peculiar name or symbolic link or tries to install to a RAID device.

 

You can explicitely specify mkfs.vfat -F32 or -F16 to define the FAT bitness.

 

To a point, yes; but:

 

• You can't create a FAT32 filesystem on a device that's smaller than 66,586 sectors (about 32.5 MiB). That's pretty tiny, of course, but if you wanted to keep the UEFI DUET files separate from the EFI System Partition for some reason, you might want a tiny partition for UEFI DUET alone.
  
• You might want to live with whatever you've got because it already exists and holds files.
  

 

FWIW, I've modified my script to use FAT32 whenever it creates a filesystem on a partition larger than 66,586 sectors and to let mkdosfs decide what to do for smaller partitions.

 

I'm unlikely to make significant additional changes for several days, at least; I've got too much else on my plate this week.

[Keshav Amburay]

I've uploaded a new version and updated the link above; but to repeat it so you don't need to find it, it's:

 

http://www.rodsbooks.com/duet-install-0.1.1.tgz

 

Regarding your installation script, I've modified mine to use it, but I've discovered one issue: Because you use the sudo command in the script, it won't work when run from PartedMagic, which doesn't include sudo. It'd be nice to be able to run it from PartedMagic, so as to prepare a hard disk before installing an OS on it.

 

I removed sudo usage and added check for root.

 

For me, 2.3 doesn't work at all on one of my computers, but 2.1 does. (I wish it were the other way around, obviously!) 2.3 crashes with a page fault or illegal instruction or some such message.

 

Thats strange. Since the old "2.1" version was the one with which you had the weird keyboard problems.

 

I have uploaded a newer build of EFI_DUET with your duet-install 0.1.1 in the git repo. Please test it. Thanks for your script.

[srs5694]

I've just tested it, and the new version with my script works for me. It's necessary to manually specify the paths to BootDuet and UEFI DUET, though, or edit them appropriately. I just realized that you include BootDuet in UEFI DUET, so it's possible to do this on lines 22-23:

 

uefiDuetPath=~/bootduet/skodabenz-EFI_DUET-157945c
bootDuetPath=$uefiDuetPath/BootSector

 

In fact, if the user doesn't move the script from the skodabenz-EFI_DUET-{version} directory, those two lines can be changed to:

 

thisScript=$(readlink -f $0)
uefiDuetPath=`dirname $thisScript`
bootDuetPath=$uefiDuetPath/BootSector

 

Those changes then obviate the need to use the -b and -u options or edit the script to customize the locations. Using -s might still be necessary, although the default setting is where it goes on every distribution I've tried if you install the syslinux package.

 

FWIW, GitHub seems to strip the executable bit from scripts -- or was that your decision?

[Keshav Amburay]

I've just tested it, and the new version with my script works for me. It's necessary to manually specify the paths to BootDuet and UEFI DUET, though, or edit them appropriately. I just realized that you include BootDuet in UEFI DUET, so it's possible to do this on lines 22-23:

 

uefiDuetPath=~/bootduet/skodabenz-EFI_DUET-157945c
bootDuetPath=$uefiDuetPath/BootSector

 

In fact, if the user doesn't move the script from the skodabenz-EFI_DUET-{version} directory, those two lines can be changed to:

 

thisScript=$(readlink -f $0)
uefiDuetPath=`dirname $thisScript`
bootDuetPath=$uefiDuetPath/BootSector

 

Those changes then obviate the need to use the -b and -u options or edit the script to customize the locations. Using -s might still be necessary, although the default setting is where it goes on every distribution I've tried if you install the syslinux package.

 

But please do not remove the -b and -u options in the script.

 

FWIW, GitHub seems to strip the executable bit from scripts -- or was that your decision?

 

I maintain the repo in a ntfs-3g mounted partition (with exec permissions) and filemode = false set in the git repo .config file. So maybe git push ignores the filemode or github does. I think its the first one.

[migle]

And as a small token of my appreciation, I've solved your EFI Windows 7 USB stick issue. Do the following:

 

1. Format your USB stick using the FAT32 format (an unpartitionned "superfloppy" works fine);

 

2. Install Bootduet on it;

 

3. Copy EFILDR20 to the root;

 

4. Copy the entire content of the Windows 7 x64 install DVD to the USB stick;

 

5. In folder EFI, create a folder named BOOT;

 

6. In that folder, copy the BOOTX64.EFI file that I included in this post.

 

And voila ! A perfectly functional EFI Windows 7 installation USB stick. I tested it and it works. I read your previous post and the problem was that the BOOTX64.EFI file included in the EFISYS.BIN and EFISYS_NOPROMPT.BIN don't work. They're only for the el-torito partition of a bootable DVD. The one I included in my post comes directly from the EFI system partition of an already installed and working Windows 7 x64 installation on an EFI GPT disk.

 

I was able to install Vista with the BOOTX64.EFI you sent from Windows 7. I didn't even bother to look for the corresponding file in the Vista DVD.

 

It also wasn't necessary to install Duet on the pen drive: I already had it on the hard disk, and because it looks for bootable removable media, it immediatly booted the BOOTX64.EFI file without even asking.

 

It didn't work the first time: I had to sort my partitions and create a Microsoft Reserved partition of 128MB (which I still don't know if it is actually in use). Then it worked.

 

Since then, I installed windows some four times: I'm ending up with a system which gives errors when I try to install some of the updates. However, I think that is related to the order by which I install the updates and the software for my laptop, and not related to using DUET. I'll try a fifth time one of these days.

 

Thanks,

 

Keshav and migle, you're both welcome to include this script in your packages. IMHO, a single binary package with everything needed to install UEFI DUET on a hard disk would help make it a viable tool for people who need it. Unfortunately, my script makes heavy use of Linux-specific programs and features, so it's not likely to be very portable. (I'm not even sure how portable it is across Linux distributions -- so far I've tested it only on Gentoo.)

 

Naturally, I welcome any comments you might have.

 

Greetings!

This is all good to hear. We're not far from turning DUET into something like EUET, Enthusiast's UEFI Emulation instead of Developer's :-)

 

I will only keep on github my source file. What I would like would be to completely tame the compilation process, which is too complicated for people that are not firmware developers, maybe taking advantage of those GNU EFI tools which are used to compile ELILO.

 

I think you're right by not changing partitions, boot flags, etc. That would only make new limitations arise. Partitioning is the job of gdisk and similar tools. The model would be GRUB. About compiling, I don't know... you use Gentoo, you know: picking things from their source repositories, patching, compiling, is pretty practical.

 

I'll comment about FAT bitness next.

 

 

What is it about those files that's filesystem-specific? That is, is there something about how they begin to load themselves that makes assumptions about the filesystem, or do they assume they're running from a particular filesystem after they've been booted, or what? If you think problems are likely to result, even on some systems, from using a FAT32-specific build on a FAT16 filesystem, I recommend building both (and FAT12, too, for completeness), since it's hard to guarantee that the target filesystem will use a particular FAT level.

 

My guess is that only the first 512 bytes of EFILDR, EFILDR16 and EFILDR20 is FAT bitness specific.

And because of that, and because that FAT loading code has all those limitations we detected, my BootDuet jumps over those first 512 bytes. I squeezed more general FAT code into the boot program than there was on DUET's bootsect.S and start32.S combined.

That's why I jump to 2000:0200 instead of 2000:0000.

 

After the entire flash image (the EFILDR file) is loaded, any access to a file system is for looking for boot programs, and that is using the FAT driver within the EFILDR image (which surely supports all three types of FAT).

 

This is my guess, because I never had the EFILDR and EFILDR16 images with me. Only the EFILDR20 that Keshav compiled (call it the law of the least effort, or the principle of minimum energy...).

I'm convinced that if you take the first 512 or maybe 4096 bytes of those three files, the rest of the files are byte for byte equal.

 

From the beginning I have done what you did: renamed EFILDR20 to EFILDR16 to use it on a FAT16 file system.

 

The ESP partition on my laptop is 16-bit too, because that's the dosfstools default for a 128MB filesystem, and it's the more logical for that size, because less space is consumed by the File Allocation Table, and it's also the safest, because it's troublesome to distinguish between FAT file system versions (there isn't a field in the BPB saying the bitness of the FAT), and some tools distinguish between FAT 12 and FAT 16 by number of sectors in the file system.

 

Maybe some people were put away from FAT12 and FAT16 by a bug I left in the code after making some changes. At this moment FAT16 surely works well.

 

I think the best approach is formating the filesystem using default parameters and then installing the correct version of BootDuet for it.

Even better: don't format the filesystem, let the user decide what files he keeps in there, just check the hidden sectors field. This is just as you let the user create the partitions, etc. Partitioning and formatting automatically only makes sense for the original use of DUET which was on a dedicated pen drive, you can never make those kinds of assumptions about someones personal harddisk.

 

Because we jump over the first 512 bytes of EFILDR, it also no longer makes sense to use a different file name for each FAT bitness. We should just use EFILDR...

When we completely take hold of the compilation process, we can also produce an image without those 512 bytes.

 

By the way, I also tried to make BootDuet support all three types of FAT bitness, but I just couldn't. I'd have to use only the 420 bytes available for a FAT32 boot program... I couldn't even squeeze the FAT12 version into 420 bytes.

So, the BootDuet image is the only thing that is FAT bitness specific.

[Keshav Amburay]

What I would like would be to completely tame the compilation process, which is too complicated for people that are not firmware developers, maybe taking advantage of those GNU EFI tools which are used to compile ELILO.

 

This is my guess, because I never had the EFILDR and EFILDR16 images with me. Only the EFILDR20 that Keshav compiled (call it the law of the least effort, or the principle of minimum energy...).

I'm convinced that if you take the first 512 or maybe 4096 bytes of those three files, the rest of the files are byte for byte equal.

 

http://dl.dropbox.com/u/9710721/UEFI/DUET_...LE_BUILD.tar.xz

http://dl.dropbox.com/u/9710721/UEFI/DUET_...LE_BUILD.tar.xz

 

Compiled files compressed with tar and (xz -9) as on 13-JUN-2011 , the (DUET_EMUVARIABLE_BUILD.tar.xz)/FV/Efildr20 is same as Efildr20 in EFI_DUET and ditto for FSVARIABLE build.

 

@migle: All the Efildr files plus the relevant scripts copied into the dir for you to see.

 

My build process :-

 

1) Set up symlinks outside (OUTER_DIR) the edk2 and other tianocore git checkout dirs - https://github.com/skodabenz/My_Shell_Scrip.../my_symlinks.sh

 

2) Then apply https://github.com/skodabenz/EFI_DUET/blob/...Makefiles.patch into EDK2 root dir - not all the changes are needed for DuetPkg, some of them are for OvmfPkg (QEMU and VirtualBox UEFI firmware), but those changes are untested.

 

For now I maintain the patch in the form of a separate git branch keshav_pr in the edk2 git repo so my duet compile script simply checks out that branch before compile, instead of applying the patch like normally needed.

 

3) Run (OUTER_DIR)/duet_x64_edk2_linux_compile_setup.sh ( https://github.com/skodabenz/My_Shell_Scrip...ompile_setup.sh ) which sources https://github.com/skodabenz/My_Shell_Scrip..._duet_common.sh which in turn sources https://github.com/skodabenz/My_Shell_Scrip...ocore_common.sh .

 

See the scripts on how the compile process occurs. Important info - I compile in Archlinux x86_64 system with gcc-multilib 4.6 (mostly latest snapshot - Archlinux is bleeding-edge like), and Archlinux uses default python to python3, so i temporarily re symlink python2 into python for the compile process to continue. Also with -Werror compile fails in gcc 4.6, that is removed in CORRECT_WERROR() in tianocore_common.sh .

 

4) duet_x64_edk2_linux_compile_setup.sh calls http://tianocore.git.sourceforge.net/git/g...ld64.sh;hb=HEAD which automates most of the compile process. No need for mingw or mingw-w64. Although it can also be compiled in Windows with VS2008 or VS2010, I am more comfortable in linux due to the ability to automate the task using bash scripts.

 

5) DuetPkg/build64.sh simply builds only the FirmwareVolume (DUET_EMUVARIABLE_BUILD.tar.xz)/FV/DUETEFIMAINFV.Fv file (don't know what that means exactly).

 

DuetPkg/PostBuild.sh ( http://tianocore.git.sourceforge.net/git/g...uild.sh;hb=HEAD ) then creates the actual Efildr files using the pre-compiled ASM Bootsector files from DuetPkg/BootSector/bin ( http://tianocore.git.sourceforge.net/git/g...bc0c4fe;hb=HEAD )

 

6) DuetPkg/PostBuild.sh also creates the memdisk file i include in the other two repos in github by calling DuetPkg/CreateBootDisk.sh in the very end ( http://tianocore.git.sourceforge.net/git/g...Disk.sh;hb=HEAD ). I also added a https://github.com/skodabenz/My_Shell_Scrip..._memdisk_old.sh file to my scripts to do this separately if the precompiled files are already available which may optionally be called in post_duet_x64_compile.sh (which I will describe later).

 

I tried to do the same memdisk but with BootDuet instead of edk2's bootsectors in https://github.com/skodabenz/My_Shell_Scrip..._memdisk_new.sh but have not succeeded sop far due to device naming assumptions in srs5694's duet-install which do not work for a superfloppy loop device.

 

7) If the above compile succeeds and the Efildr files are generated successfully the i manually run https://github.com/skodabenz/My_Shell_Scrip..._x64_compile.sh which copies the files ot the relevant local location of the github repos, commits the changes with the DATE as the commit messages. I push the changes to github manually though.

 

The main reason sometimes Efildr files are not generated are due to some filesize restriction hardcoded in http://tianocore.git.sourceforge.net/git/g...nPage.c;hb=HEAD (mainly #define EFI_PAGE_BASE_OFFSET_IN_LDR 0x70000 ). This file is called in DuetPkg/PostBuild.sh for X64 firmware alone.

 

I had to disable some modules in http://tianocore.git.sourceforge.net/git/g...X64.dsc;hb=HEAD (file that includes modules to be compiled) and http://tianocore.git.sourceforge.net/git/g...Pkg.fdf;hb=HEAD (file that defines modules that should go into FV image) . See the UDK_EDK2_DuetPkg_Changes_to_Makefiles.patch patch for the changes I made to those files. If someone know what GenPage exactly does and can circumvent the filesize restriction I can include more modules like floppy (right nor the Efildr file will not recognize a floppy disk drive (hardware) once loaded, this restriction does not apply to loading Efildr from a floppy disk using BootDuet).

 

For all these scripts checkout https://github.com/skodabenz/My_Shell_Scripts . Hope this helps.

 

Also gnu-efi http://sourceforge.net/projects/gnu-efi/ headers which ELILO uses, are a very minimal subset of efi-toolkit http://sourceforge.net/apps/mediawiki/tian...tle=EFI_Toolkit project which was active when EDK1 was used. Now EDK2 is one big integrated UEFI development environment which is much more updated and follow the latest Spec version. http://tianocore.git.sourceforge.net/git/g...=StdLib;hb=HEAD is the EDK2 equivalent of efi-toolkit.

[migle]

@migle: All the Efildr files plus the relevant scripts copied into the dir for you to see.

 

Ok, I just checked those files to see if EFILDR, EFILDR16 and EFILDR20 are equal except for the first 512 bytes. They are. That means that when using BootDuet we can safely just rename the file, as we have been doing, because it skips those first 512 bytes, and that means there's no point in keeping separate copies of those.

 

For all these scripts checkout https://github.com/skodabenz/My_Shell_Scripts . Hope this helps.

 

More than compiling Duet, I would like to understand it. I know that EFI files are PE executable files, that's good, we know all about those, they're COFF based, generated by some linker, obviously they're static executables that don't export symbols, GNU ldd will generate them given appropriate parameters.

 

Then there's FV files, Firmware Volume. Ok, they're something you would write on a flash memory or something. They're generated by that GenFV program given an INF file (which is a kind of makefile). But what are they? Collections of EFI files? FV files are described in the UEFI specification, but I didn't look into that yet.

 

Then there's FW files? What are those? Are they actually used here? It would be nice to strip this thing of things we don't need.

 

There's the EFILDR image, generated by EfiLdrImage. Reading the comments, it's a header (which we already know contains the loading code and the code that switches to protected mode and then to 64 bit mode), plus a collection of PE32 executables (EFI files? FV files?)...

 

You see, on the bootsect directory there's a readme file with a nice diagram telling how the multiple source files are combined. I would like to understand this at that level.

 

Then there's that whole bunch of GenThis, GenThat files. Some of them are of no use to us, who use Linux. On Windows, those simple tools like dd aren't available, so simply combining images of parts of images together require that a special purpose tool is written. We don't need those, and anyone who wants to use DUET for the same purpose as we do doesn't need them either (only a masochist would use DUET for booting Windows if it's not for having Linux or FreeBSD on the same machine).

 

 

Also gnu-efi http://sourceforge.net/projects/gnu-efi/ headers which ELILO uses, are a very minimal subset of efi-toolkit http://sourceforge.net/apps/mediawiki/tian...tle=EFI_Toolkit project which was active when EDK1 was used. Now EDK2 is one big integrated UEFI development environment which is much more updated and follow the latest Spec version. http://tianocore.git.sourceforge.net/git/g...=StdLib;hb=HEAD is the EDK2 equivalent of efi-toolkit.

 

I just looked at that link you posted. Good, it's a runtime library, I understand that and it's essential for building a free-standing executable. I see the INF file for the dreaded python build tool. If I didn't have the python build tool, I would be able to do a Makefile for compiling that into a static library.

 

I'll take it for granted that that library is better than the gnu-efi. It even has floating-point support functions.

 

This is the kind of thing that is needed to understand this. To separate this into components are understand each of them in isolation. You already went far in that, but I didn't yet have the time for it.

 

We need to have a minimal understanding of how these files (FV, EfiLdr) are composed to be able to (as you are already able, I think) pick modules from other projects (VirtualBox). And we'll need to understand how this thing lays in memory and how come the OS doesn't overwrite it if we ever want to replace one module (such as FSVariable).

 

Of course, "need" is relative. This thing already does more or less what we want. It's still not the perfect pre-boot environment, but is already good enough to boot a single operating system (Windows) and we really don't need to boot any other.

[srs5694]

But please do not remove the -b and -u options in the script.

 

No, of course not. FWIW, I've uploaded a slightly more dynamic version of this change to:

 

http://www.rodsbooks.com/duet-install-0.1.2.tgz

 

This version checks for the presence of the copy_duet_files.sh file in the same directory as duet-install. If that file is present, the script sets the uefiDuetPath variable to the script's directory. If not, it sets it to ~/bootduet/skodabenz-EFI_DUET -- perhaps not better than the duet-install script's directory, but maybe it is. (It is for me.) The -u option overrides either of these, of course, and -b sends the script looking for BootDuet files elsewhere, too.

 

I think you're right by not changing partitions, boot flags, etc. That would only make new limitations arise. Partitioning is the job of gdisk and similar tools. The model would be GRUB. About compiling, I don't know...

 

I named my script "duet-install" after "grub-install." It actually does change boot flags, and sometimes partition type codes (it changes type codes when the -F option is used to create a FAT filesystem). As to compiling, since Keshav includes BootDuet binaries in his package set, and since binaries of SYSLINUX come with it, those who aren't familiar with software compilation can install everything. I've tested it from PartedMagic and it works there.

 

I think the best approach is formating the filesystem using default parameters and then installing the correct version of BootDuet for it.

 

One problem I ran into is that the Windows 7 installer flakes out if the ESP uses FAT16. (Probably FAT12, too, but I didn't test it.) Thus, if you want to use the ESP as the target for UEFI DUET and install Windows, it's best to use FAT32. This probably wouldn't be an issue if you were to use a separate ESP from the UEFI DUET installation partition, though. That's one of the reasons I changed my script to favor FAT32 whenever possible.

 

Partitioning and formatting automatically only makes sense for the original use of DUET which was on a dedicated pen drive, you can never make those kinds of assumptions about someones personal harddisk.

 

It can also make sense if you're setting everything up from scratch, although I admit that creating a filesystem separately isn't a big deal. I chose to make the default in my script to not create a filesystem, since I didn't want it accidentally trashing something important; but with the -F option, it will do so.

 

only a masochist would use DUET for booting Windows if it's not for having Linux or FreeBSD on the same machine

 

Actually, there's one potentially important reason for using UEFI DUET in a Windows-only environment: Getting access to >2 TiB hard disks. Of course, it's possible to use such a disk as a data (non-boot) disk without UEFI, but to use all of such a disk you need GPT, and to boot Windows from a GPT disk you need UEFI.

 

I've spoken with engineers at a couple of disk manufacturers, and they're concerned about this issue. (Or they were a few months ago; maybe they've got some other solutions coming by now.) I figure it's only a matter of time before a Windows-only user finds this thread because he's desperate to get his new 3 TB disk working properly in his small-form-factor computer that has room for just one hard disk.

 

Incidentally, does anybody know how to turn the recovery DVDs created from a recovery partition on a Windows 7 computer into a more standard install DVD? I ask because my laptop with Windows 7 Home has these recovery DVDs, but they include no EFI boot option, and I'm trying to get something that I can use to re-install Windows in EFI mode.

 

Edit: I just did a Web search and I found, surprisingly, that you can download legal Windows installation DVD images from here. You've got to have a valid license key to use them, of course, and license keys are tied to specific product versions, so this isn't a loophole for a free upgrade!

[Keshav Amburay]

One problem I ran into is that the Windows 7 installer flakes out if the ESP uses FAT16. (Probably FAT12, too, but I didn't test it.) Thus, if you want to use the ESP as the target for UEFI DUET and install Windows, it's best to use FAT32. This probably wouldn't be an issue if you were to use a separate ESP from the UEFI DUET installation partition, though. That's one of the reasons I changed my script to favor FAT32 whenever possible.

 

I don't know whether it is actually true, haven't found any documentation regarding this. But I never tried FAT16 or FAT12 (which will be never be actually used since EFISYS should be atleast 100 MiB) for EFISYS partition. I used mkfs.vfat -F32 option for my EFISYS and my DUET partition.

 

Incidentally, does anybody know how to turn the recovery DVDs created from a recovery partition on a Windows 7 computer into a more standard install DVD? I ask because my laptop with Windows 7 Home has these recovery DVDs, but they include no EFI boot option, and I'm trying to get something that I can use to re-install Windows in EFI mode.

 

Edit: I just did a Web search and I found, surprisingly, that you can download legal Windows installation DVD images from here. You've got to have a valid license key to use them, of course, and license keys are tied to specific product versions, so this isn't a loophole for a free upgrade!

 

You can also try http://www.insanelymac.com/forum/index.php...t&p=1261601 .

 

BTW see the link in post #71 on how to create Archlinux BIOS+UEFI and Windows x64 BIOS+UEFI USB drive. It will also be useful for other UEFI supporting distros like Ubuntu, Fedora (if using grub2-efi) or Clonezilla.

[scrapdogg]

i'd like to thank keshav for posting this guide on changing win7 to gpt without formatting, if a windows monkey like myself with no linux knowledge and even less than no mac knowledge can get this working then anyone can

 

for whatever reason neither sysresccd nor partedmagic would boot to a working gui no matter what i tried on my system (win7 64bit, gigabyte ga-x58a udr3 rev2,hitachi 3tb hdd{the reason i need to boot from gpt so i canactually use the whole drive}, core i7 950, and 2 radeon 6870's in crossfire). what did actually boot up to the gui's were riplinux grub iso and the gparted iso.

 

bet you're wondering why i needed both

 

i couldn't figure out for the life of me how to add gdisk (gptfdisk) to the riplinux iso, so i had to use it from the command prompt in gparted and then boot riplinux after the mbr to gpt conversion for the other tools since riplinux has a great gui. after mustering up enough courage to try this out i finally got it working after alot of google searches for "linux command for "?" (lol i told you i suck at linux) although i'm more than comfortable at a dos prompt "go figure".

 

one more thing, (setterm -powersave off -blank 0) before you start the gui is a must with riplinux as any partition work in gparted that is going to take a while will come to a screeching halt if the screensaver kicks in (found that out the hard way at 2:00 this morning when a partition move should have been completed only to find it at 20% done... and since i couldn't run that command with the gui already started i applied good old duct tape theory to the equaiton and used a pen to hold down one of the keys on the keyboard and went back to sleep

 

after i got everything working the 200mb efi partition was in the middle of the disk between 2 ntfs partitions. imagine my horror when i couldn't extend the os partition to the rest of the unallocated space on the drive. so when you make that partition you might think about creating it at the end of the disk if you want 1 big data partition (generally frowned upon i know, but i always shrink it in windows before i back it up). or you can use gparted to move partitions around afterwards like i'm doing now.

 

one more more thing while i'm thinking about it on my particular motherboard the efi boot option will only actually boot up windows 7 on the intel sata ports!!! not the marvell ports and definitely not the jmicron ports. (you have been warned!) and this goes without saying but, USE WINDOWS IMAGE BACKUP TO BACKUP YOUR OS BEFORE YOU TRY ANY OF THIS!!!. i think i restored my mbr backup 3 times before i got everything right.

 

thanks to everyone else who added to this thread as well, hope my mistakes can help someone out who know even less about linux than i do and i fully expected the forum quizzes to be all osx stuff but to my surprise they were just general tech stuff. first quiz 8/12 second one 19/20. not bad for an old dos/windows guy. i'm actually a+ certified if you can believe that!! lol, more like a+ certifiable

[srs5694]

I don't know whether it is actually true, haven't found any documentation regarding this. But I never tried FAT16 or FAT12 (which will be never be actually used since EFISYS should be atleast 100 MiB) for EFISYS partition. I used mkfs.vfat -F32 option for my EFISYS and my DUET partition.

 

My comment about Windows flaking out with non-FAT32 ESPs is based on my own observations. I was actually getting quite frustrated because I was initially letting mkdosfs set the FAT size, and it uses FAT16 on 100-200 MiB partitions. Windows will then begin to install but complain that the partitions aren't in the recommended order. It then fails after copying a bunch of files and, on reboot, I'd find that there were two ESPs -- the one I created and the one that Windows created because it didn't like the one I created. Switching to FAT32 solves those problems. Using a separate non-ESP partition for UEFI DUET and letting Windows create the disk's ESP would also presumably work, but I've not tested that.

 

Incidentally, I've discovered another "gotcha:" Both GParted and GNU Parted (and probably other GParted-based tools) clear all the GPT attribute flags from all partitions whenever they create or delete (and maybe modify) any partition. This of course wreaks havoc with SYSLINUX, which relies on the "Legacy BIOS Bootable" flag being set. That's not so much of a problem if you're using GRUB in the MBR, but with SYSLINUX it's a pain.

 

i'd like to thank keshav for posting this guide on changing win7 to gpt without formatting, if a windows monkey like myself with no linux knowledge and even less than no mac knowledge can get this working then anyone can

 

ROTFL! Just two posts after I predicted it, a Windows-only person comes along! (Although it sounds like you've got a motherboard with built-in UEFI, so you're not using DUET.)

 

bet you're wondering why i needed both

 

i couldn't figure out for the life of me how to add gdisk (gptfdisk) to the riplinux iso, so i had to use it from the command prompt in gparted and then boot riplinux after the mbr to gpt conversion for the other tools since riplinux has a great gui.

 

FWIW, USB flash drives are great tools for situations like this. You can drop whatever you need on them and use them from the flash drive, or perhaps copy them over.

 

after i got everything working the 200mb efi partition was in the middle of the disk between 2 ntfs partitions. imagine my horror when i couldn't extend the os partition to the rest of the unallocated space on the drive. so when you make that partition you might think about creating it at the end of the disk if you want 1 big data partition (generally frowned upon i know, but i always shrink it in windows before i back it up). or you can use gparted to move partitions around afterwards like i'm doing now.

 

On a fresh installation, the usual practice is to put the ESP at the very start of the disk. If you're converting an existing setup, though, the end likely makes more sense than the middle because it won't get in the way if it's at the end. An alternative would be to shrink your Windows partition from its start to clear up space at the start of the disk, but that type of repartitioning operation is always a bit risky. (It's generally safer to modify the end point of a partition rather than the start point.)

 

one more more thing while i'm thinking about it on my particular motherboard the efi boot option will only actually boot up windows 7 on the intel sata ports!!! not the marvell ports and definitely not the jmicron ports. (you have been warned!)

 

You could check the motherboard and/or chipset manufacturers' Web sites to see if they've got UEFI drivers for their products. Even if they do, though, that might not be all that much help, since the UEFI needs to be able to read the disk in order to load the driver, and if that's your only disk, it won't help much to have a driver on a disk that the firmware can't read. One way around it might be to put the firmware in a USB flash drive or something similar. This might be worth doing if the Marvell or JMicron ports were faster or otherwise preferable to the Intel ports. If not, though, moving the cable to an Intel port is certainly simpler.

[Keshav Amburay]

My comment about Windows flaking out with non-FAT32 ESPs is based on my own observations. I was actually getting quite frustrated because I was initially letting mkdosfs set the FAT size, and it uses FAT16 on 100-200 MiB partitions. Windows will then begin to install but complain that the partitions aren't in the recommended order. It then fails after copying a bunch of files and, on reboot, I'd find that there were two ESPs -- the one I created and the one that Windows created because it didn't like the one I created. Switching to FAT32 solves those problems. Using a separate non-ESP partition for UEFI DUET and letting Windows create the disk's ESP would also presumably work, but I've not tested that.

 

Might be related to UEFI Spec specifically mentioning FAT32 for ESP (atleast in one place in the document).

 

Incidentally, I've discovered another "gotcha:" Both GParted and GNU Parted (and probably other GParted-based tools) clear all the GPT attribute flags from all partitions whenever they create or delete (and maybe modify) any partition. This of course wreaks havoc with SYSLINUX, which relies on the "Legacy BIOS Bootable" flag being set. That's not so much of a problem if you're using GRUB in the MBR, but with SYSLINUX it's a pain.

 

GParted (and KDE Partition Manager) depends on libparted (part of GNU Parted) so thats the one likely creating the problem. I actually had this error with Syslinux when I repartitioned my linux system using GParted for separate /var about two months back but I didn't suspect libparted was the cause of the problem. I simply replaced syslinux with grub2 bios but didn't check the /boot partition GPT attributes.

 

BTW RIP (Recovery Is Possible, not Rest In Peace lol) linux http://www.tux.org/pub/people/kent-robotti/looplinux/rip/ includes gdisk command in the iso but I am not sure of fixparts though. I use it instead of parted-magic since parted magic does not provide a 64-bit kernel (for chroot operations). For a person new to the linux world I would recommend System Rescue CD for an ALL-IN-ONE rescue system but its download size is slightly bigger than RIP Linux or parted-magic. AFAIK RIP uses Fluxbox, SysResCD uses XFCE and Parted Magic uses LXDE as the Desktop Environment. Of these Parted Magic does not provide a 64-bit kernel.

 

Also a Microsoft Reserved Partition (msftres GNU Parted flag) is not needed to boot Windows in UEFI mode. That partition is used to compensate for lack of post MBR 31 KiB gap (due to cylinder alignment in MBR setup) in GPT table. This region is used by Windows to store partition metadata of dynamic disks within the GPT system. Dynamic disks are similar to linux LVM. Having the msftres partition allows you to convert your disk to dynamic disk anytime. If you want to prevent Windows from converting the disk to Dynamic, I suggest you delete the msftres partition.

 

http://en.wikipedia.org/wiki/Microsoft_Reserved_Partition

http://en.wikipedia.org/wiki/Logical_Disk_...dynamic_volumes

http://msdn.microsoft.com/en-us/windows/ha...201104111922443

http://msdn.microsoft.com/en-us/library/aa...28VS.85%29.aspx

[scrapdogg]

I've spoken with engineers at a couple of disk manufacturers, and they're concerned about this issue. (Or they were a few months ago; maybe they've got some other solutions coming by now.) I figure it's only a matter of time before a Windows-only user finds this thread because he's desperate to get his new 3 TB disk working properly in his small-form-factor computer that has room for just one hard disk.

 

Incidentally, does anybody know how to turn the recovery DVDs created from a recovery partition on a Windows 7 computer into a more standard install DVD? I ask because my laptop with Windows 7 Home has these recovery DVDs, but they include no EFI boot option, and I'm trying to get something that I can use to re-install Windows in EFI mode.

 

Edit: I just did a Web search and I found, surprisingly, that you can download legal Windows installation DVD images from here. You've got to have a valid license key to use them, of course, and license keys are tied to specific product versions, so this isn't a loophole for a free upgrade!

 

if you do end up downloading one of those legal windows isos, you can force it to uefi mode with the information in this thread, specifically keshav's post 2nd down from the top (keshav to the rescue again!!). i actually had to do it myself so i know that it works

 

http://www.insanelymac.com/forum/index.php?showtopic=184349

 

and lol you called it alright, although i actually have 3 hdd's in my computer WOOOOHOO go me!!

 

if i had known then what i know now i woulda bought a 2tb hdd just kidding i love breaking stuff and putting it back together!!! that's how i got the nickname "doc" as a kid i would fix stuff even if it wasn't broken!

 

and keshav you are absolutely right riplinux does in fact have gdisk and it also has fixparts as well. didn't even think to check since i had already burned gparted to a cdrw (there's one i can erase and get back) lol. i actually tried system rescue cd but like i said earlier my dumb couldn't get the gui part to show up on my system. an id10t error i'm sure.

 

thanks again guys

[migle]

Actually, there's one potentially important reason for using UEFI DUET in a Windows-only environment: Getting access to >2 TiB hard disks. Of course, it's possible to use such a disk as a data (non-boot) disk without UEFI, but to use all of such a disk you need GPT, and to boot Windows from a GPT disk you need UEFI.

 

I've spoken with engineers at a couple of disk manufacturers, and they're concerned about this issue. (Or they were a few months ago; maybe they've got some other solutions coming by now.) I figure it's only a matter of time before a Windows-only user finds this thread because he's desperate to get his new 3 TB disk working properly in his small-form-factor computer that has room for just one hard disk.

 

You're right, it's true, I forgot that. And they're selling older BIOSes even today! For many years to come people will want to have their >3TB or >30TB :-) disks and it's important that motherboards, processors, etc, don't simply become garbage because of such a minor issue (not minor... a little thing with a big impact).

[Keshav Amburay]

My comment about Windows flaking out with non-FAT32 ESPs is based on my own observations. I was actually getting quite frustrated because I was initially letting mkdosfs set the FAT size, and it uses FAT16 on 100-200 MiB partitions. Windows will then begin to install but complain that the partitions aren't in the recommended order. It then fails after copying a bunch of files and, on reboot, I'd find that there were two ESPs -- the one I created and the one that Windows created because it didn't like the one I created. Switching to FAT32 solves those problems. Using a separate non-ESP partition for UEFI DUET and letting Windows create the disk's ESP would also presumably work, but I've not tested that.

 

The below quote is not of any forum post. It is copied form the UEFI Specification 2.3.1 (supported by UDK/EDK2 DUET) Chapter 12.3 File System Format.

 

EFI encompasses the use of FAT32 for a system partition, and FAT12 or FAT16 for removable media. The FAT32 system partition is identified by an OSType value other than that used to identify previous versions of FAT. This unique partition type distinguishes an EFI defined file system from a normal FAT file system. The file system supported by EFI includes support for long file names.

 

FAT defines that all files in a directory must have a unique name, and unique is defined as a case insensitive match.

 

UEFI does not impose a restriction on the number or location of System Partitions that can exist on a system. System Partitions are discovered when required by UEFI firmware by examining the partition GUID and verifying that the contents of the partition conform to the FAT file system as defined in Section 12.3.1.1. Further, UEFI implementations may allow the use of conforming FAT partitions which do not use the ESP GUID. Partition creators may prevent UEFI firmware from examining and using a specific partition by setting bit 1 of the Partition Attributes (see 5.3.3) which will exclude the partition as a potential ESP.

 

Also see - Microsoft EFI FAT32 File System Specification - http://msdn.microsoft.com/en-us/windows/ha...e/gg463080.aspx .

[migle]

The below quote is not of any forum post. It is copied form the UEFI Specification 2.3.1 (supported by UDK/EDK2 DUET) Chapter 12.3 File System Format.

 

Hmm. Well, Vista installed fine with a FAT16 ESP.

 

Sometimes, people overspecify things with no good reason. In the end, the firmware will have to support all three types even if just for removable media. That is to say, specifying that, will not relieve the firmware from dealing with the complexity of the three types of FAT. Also, an implementation which followed that paragraph strictly would have to have additional complexity for deciding which media is removable and limiting the types of FAT that would be recognized.

 

But, that quote is reason enough for "pushing" FAT32 to people by default.

 

Also see - Microsoft EFI FAT32 File System Specification - http://msdn.microsoft.com/en-us/windows/ha...e/gg463080.aspx .

 

Hmm..... Following that link, what caught my attention was the license for the specifcation... It brings to mind that FAT32 now is a patented proprietary file system that has a specification which is available only to those who accept an agreement. Really. The only reason for that document being there is for people accepting the licence.

 

Who needs that spec? Everyone knows how FAT32 works: it's just like FAT16 (unpatentable common knowledge) except for one or two minor differences. Just read it on Wikipedia. LFN where a more complex change. There's open source software for FAT32 for at least 15 years. I'm really sorry that digital camera manufacturers didn't get together and challenge Microsoft's claims legally.

 

And who approved the UEFI spec fell into forcing the use of a proprietary file system...

 

That's also a good reason for having FAT16 and 12 as an option.

 

Thanks for looking it up in the UEFI spec.

[srs5694]

The FAT bitness thing gets worse: There a bug in Ubuntu 11.04 that causes it to trash existing ESPs, at least if they're FAT32 (I don't know if it also trashes FAT16 ESPs), replacing them with FAT16 ESPs. This imposes a hurdle for the coexistence of Windows and Ubuntu on a UEFI system: If you start with a Windows installation, Ubuntu trashes the ESP and makes Windows unbootable; and if you start with Ubuntu, Windows can't complete its installation and creates a duplicate ESP (if there's room on the disk). Either way, the only solution I'm aware of is to back up the ESP and restore it at one point or another.

[srs5694]

FWIW, I've put up some documentation on installing and using UEFI DUET on my Web page:

 

http://www.rodsbooks.com/bios2uefi/

 

It's geared toward intermediate-to-advaned Windows users, although the Linux-specific sections raise the bar a bit, the assumption being that Linux users need less hand-holding when it comes to mounting volumes, etc.

[Keshav Amburay]

I have created a mirror of the git repos at gitorious - https://gitorious.org/tianocore_uefi_duet_builds . Since I use github for my personal stuff (and it has a 300 MB or MiB limit), I thought gitirious is more suited for stuff like duet.

 

FWIW, I've put up some documentation on installing and using UEFI DUET on my Web page:

 

http://www.rodsbooks.com/bios2uefi/

 

It's geared toward intermediate-to-advaned Windows users, although the Linux-specific sections raise the bar a bit, the assumption being that Linux users need less hand-holding when it comes to mounting volumes, etc.

 

Few comments :-

 

UEFI DUET currently lacks support for common optical disc filesystems (ISO-9660 and UDF). This doesn't prevent you from installing an OS, perhaps even from such a disc; but you must have a way to start the installer from a hard disk partition or USB flash drive.

 

UEFI can boot CD/DVDs just fine provided they come with a EFI floppy image and a eltorito 0xEF entry pointing to that floppy image in the iso. which will be mounted by the UEFI firmware as a virtual EFISYS filesystem.

 

UEFI DUET doesn't work on all computers. Most importantly, it's really only useful on 64-bit x86-64 computers, especially in binary form. Even on such systems, it doesn't start up properly on all computers.

 

I am not so well-versed with English language but this sound like DUET does not start in ALL x86_64 computers. I think it should be 'in some of the systems'.

 

Please add link to gitorious page for the duet binaries https://gitorious.org/tianocore_uefi_duet_builds . I may even use it as default if i plan to change my github account to 'for personal stuff only' .

 

Extract the 1/Windows/Boot/EFI/bootmgfw.efi file from the SOURCES/install.wim file on the Windows installation disc. This file is in 7zip format, so you'll need a suitable unarchiver. I used 7z under Linux.

 

It is actually a Microsoft specific format called http://en.wikipedia.org/wiki/Windows_Imaging_Format . 7-zip aka p7zip is the only non-Microsoft decompressor tool that can (as of now) extract WIM files.

 

This isn't a DUET issue specifically, but because Linux and Windows use the same partition type GUID to identify their filesystem partitions, Windows will see Linux filesystem partitions as unformatted Windows partitions; they'll show up in the Computer window and, if you click them, Windows will prompt you to format them. This is a disaster waiting to happen. I recommend you change the type code of Linux filesystem partitions using gdisk. Using a type code of 8301 ("Linux reserved") should keep Windows from messing with your Linux partitions.

 

I don't recommend Linux Reserved GUID either. We don't know (yet) for what purpose the linux kernel uses the partition type. Its better to stick with some random GUID untill a linux specific Data partition GUID is defined by upstream.

 

This link can go into the Additional Resources section http://homepage.ntlworld.com/jonathan.debo...ot-process.html . Also like [url="http://www.insanelymac.com/forum/topic/279450-why-insanelymac-does-not-support-tonymacx86/"]#####[/url] XPC seems to be another DUET based hackintosh bootloader.

 

 

 

 

And http://homepage.ntlworld.com/jonathan.debo...ot-process.html too.

[srs5694]

I have created a mirror of the git repos at gitorious - https://gitorious.org/tianocore_uefi_duet . Since I use github for my personal stuff (and it has a 300 MB or MiB limit), I thought gitirious is more suited for stuff like duet.

 

One problem with that site is that there's no obvious way to download the whole package as a tarball or a ZIP file. Using git may be OK for people who are already familiar with it, but it's a very specialized tool, and lots of people aren't familiar with it. If there's something I've overlooked, could you please point it out to me?

 

UEFI can boot CD/DVDs just fine provided they come with a EFI floppy image and a eltorito 0xEF entry pointing to that floppy image in the iso. which will be mounted by the UEFI firmware as a virtual EFISYS filesystem.

 

Interesting, but apparently of only theoretical interest -- I ran some tests, and I was unable to get Fedora 15, OpenSUSE 11.4, Ubuntu 11.04, or Windows 7 to boot using the optical disc option in the UEFI DUET Boot Manager. If it works for you, there must be some trick to it, or maybe it only works on some systems. If so, I'd appreciate hearing precisely how you got it to work. Thanks.

 

UEFI DUET doesn't work on all computers.

I am not so well-versed with English language but this sound like DUET does not start in ALL x86_64 computers. I think it should be 'in some of the systems'.

 

To a native speaker, at least of American English, the construction I used clearly means that it works on some but not all computers. Nonetheless, I've changed the wording since clearly not everybody reading the page will be a native English speaker.

 

It is actually a Microsoft specific format called http://en.wikipedia.org/wiki/Windows_Imaging_Format . 7-zip aka p7zip is the only non-Microsoft decompressor tool that can (as of now) extract WIM files.

 

Thanks for the correction; I've modified the page appropriately.

 

I don't recommend Linux Reserved GUID either. We don't know (yet) for what purpose the linux kernel uses the partition type. Its better to stick with some random GUID untill a linux specific Data partition GUID is defined by upstream.

 

AFAIK, the kernel doesn't use partition type codes at all; they're only used by partitioning tools and other user-space programs. Still, I do appreciate your point, but any way you do it it's a bad choice. I'll try contacting the libparted people to see if I can coordinate creation of a new code for Linux data partitions....

 

This link can go into the Additional Resources section http://homepage.ntlworld.com/jonathan.debo...ot-process.html . Also like [url="http://www.insanelymac.com/forum/topic/279450-why-insanelymac-does-not-support-tonymacx86/"]#####[/url] XPC seems to be another DUET based hackintosh bootloader.

 

Thanks for the links. FWIW, I considered adding XPC, but I couldn't find anything remotely resembling an authoritative URL for it; like too many things in the Hackintosh world, XPC just seems to be floating around out there on various download sites. If you know of an authoritative reference for XPC, I'll add it, but I don't want to add a link to some random download site that might go away or become a link to something obsolete in a week.

 

BTW, I've done some more testing, and I've had success with UEFI DUET on only two of four computers for version 2.1 and only one of four computers for 2.3. The numbers are small, but one pattern is that the only computer to work with both versions uses an Intel CPU; my other three test computers all have AMD CPUs. That makes me wonder if you might have set an Intel-specific optimization when you compiled the code. If so, unsetting that option would make sense. Of course, it's also possible that the AMD/Intel correlation I'm seeing here is just a coincidence, or there could be something intrinsic to the code itself, rather than compile options, that makes it work poorly or not at all on AMD systems.

[Keshav Amburay]

@srs5694: In the bios2uefi page you mention thet grub2 seems to be most difficult-to-setup UEFI bootloader. Can you elaborate on it? Can you try https://github.com/skodabenz/My_Shell_Scrip...2/grub2_uefi.sh ?

 

For others: rEFIt is written specifically for Intel Macs, not for any UEFI system, as a (kinda) multiboot boot-loader, so you may have problems with it in DUET or in Sandy Bridge UEFI systems.

[Keshav Amburay]

@srs5694: Please change the UEFI DUET link in bios2uefi page to https://gitorious.org/tianocore_uefi_duet_b...-tarball/master . This link works. I plan to remove github repos soon.

 

For updated links see http://www.insanelymac.com/forum/index.php...t&p=1300176 .

[Dr. Hurt]

How about installing Windows 7 x64 on a real UEFI system. Does windows default to using the BIOS compatibility layer, or does it detect the UEFI and boot, format HDD accordingly. I'm getting a Dell inspiron N5110 soon which comes with APTIO UEFI. Really curious as to how this'll go.

[Laqk]

How about installing Windows 7 x64 on a real UEFI system. Does windows default to using the BIOS compatibility layer, or does it detect the UEFI and boot, format HDD accordingly. I'm getting a Dell inspiron N5110 soon which comes with APTIO UEFI. Really curious as to how this'll go.

In order to install Windows 7 x64 in true UEFI mode, the install DVD needs to have the proper el-torito record. This is the case with genuine Microsoft DVDs, or DVDs created by burning a genuine Microsoft ISO image. DVDs created with customized Windows 7 installs from applications like rt7lite will not work because they don't create the proper el-torito header.

 

There is a way to manually create an ISO image containing the proper el-torito records using the oscdimg.exe tool from either OPK or WAIK. In fact, that's the tool Microsoft use the create their own DVD images. That's what I used to create my own custom install Windows 7 x64 DVD and it worked very well, allowing me to install Windows 7 x64 in UEFI mode on a GPT-partitionned HDD.

[Keshav Amburay]

There is a way to manually create an ISO image containing the proper el-torito records using the oscdimg.exe tool from either OPK or WAIK. In fact, that's the tool Microsoft use the create their own DVD images. That's what I used to create my own custom install Windows 7 x64 DVD and it worked very well, allowing me to install Windows 7 x64 in UEFI mode on a GPT-partitionned HDD.

 

http://www.insanelymac.com/forum/index.php...t&p=1261601