152 replies to this topic

[Keshav Padram]

balta2ar said:

I didn't have a CD/DVD drive so I had to prepare a bootable copy of Windows on a USB pen. I tried different programs but finally I ended up with UltraISO. However, when I tried to boot widows from USB, installer refused to accept my partitions. Many encountered this message when windows says that disk is GPT-formatted. That means that installer was run not in EFI mode.

Although this is technically not related to linux, setting up Windows 7 x64 iso to boot from USB for a BIOS boot sets up the USB as NTFS, not FAT32. Thats why the firmware fall-back to BIOS boot since it did not find any UEFI bootable disk/drive .

Quote

In parallel, I also tried to install ArchLinux from another USB stick. I burned latest archboot-2011.05 and followed the standard installation procedure. At the end I choose grub2-efi-x64. However, I didn't see a way to boot it automatically. The only way I could boot grub2 is from EFI shell. That was the first glimmer of hope. Unfortunately, I was dependent on USB srick and couldn't boot anything without it. More googling again and I resolved the problem by puttin the shell to <EFI_SYSTEM_PARTITION>/shellx64.efi. After that "Launch EFI shell" option in BIOS dropped me into shell. USB was not necessary anymore. I could boot ArchLinux from harddisk with lots of hand actions.

2. What does Windows installer do so special that "Windows boot manager" appears in the boot menu? How do I do the same for grub2? How do I add it to the BIOS Boot menu? I tried to reproduce folder structure for grub on ESP: I copied contents of grub folder to /efi/grub/boot on the analogy to /efi/microsoft/boot/ folder. It didn't work - grub didn't appear in BIOS Boot menu.

https://wiki.archlin...face#UEFI_Shell and https://wiki.archlin...re_Boot_Manager . For this to work you should have booted in UEFI mode. By standard procedure i suppose you used isohybrid (dd the iso to the usb) option and booted in BIOS mode.

Quote

I tried to launch all the efi files I could found on the windows 7 usb. Launching bootmgr.efi, memtest.efi always resulted in error sayng that the specified binary is not a recognized command or batch or a file. cdboot.efi and cdboot_noprompt.efi seemed to have started loading, but few seconds later it crashed and I was dropped back to the shell. So I couldn't install windows in EFI mode, not even run the installation. An interesting note here is that cdboot.efi and cdboot_noprompt.efi do not autocomplete in EFI shell. All other efi files do.

Finally, I managed to install Windows 7 in EFI mode by accident. Unintentionaly, I left USB DVD drive plugged in and booted into the EFI shell. My USB stick with Windows 7 was also plugged in. I was fooling around on the available drives and tried to launch EFI files I collected from different sources. During that process I run cdboot.efi from USB Windows 7 again and... surprise, surprise, installation started! It didn't drop back to the shell as previously. I also heard DVD disk spinning in the USB DVD driver! Few seconds later Windows accepted my GPT disk and I continued with installation. I went away from the PC for about ten-fifteen minutes but when I got back, Windows was already installed! How amazingly fast! It didn't recognize some of my hardware including network card, though. I rebooted and discovered "Windows boot manager" in BIOS boot menu. From now on I could boot Windows without a problem.

Let me summarize my experience in form of current problems and questions:

1. [Why can't I/How do I] boot Windows DVD in UEFI mode without all this magic with same USB stick? I saw on youtube people running the same Windows 7 in UEFI from DVD without a problem. They got UEFI: DVD in Boot menu. I don't. Why?

http://ubuntuforums....mp;postcount=76

Quote

3. Is there any document describing ASUS P8P67 specific EFI behaviour? What conditions should be met in order to boot device in UEFI mode? FAT32 Partition type, certain files availble, partition flags?

The instructions are common for any UEFI systems (excluding Intel Macs which are actually weird in UEFI booting)

Quote

4. Note: I tried to run rEFIt EFI files in the shell but failed: 'refit.efi' is not recognized as an internal or external command, operable program, or batch file.

rEFIt is specific to Intel Macs. It does not work for non-Mac UEFI systems.

Quote

5. Note: I get kernel panic when I reboot ArchLinux: http://img.flashtux....60x584317ed.jpg. Shutdown is fine, though. That's very unpleasant because I cannot reboot my PC remotely - I need to push Reset button in order to reboot the machine.

https://bugs.archlinux.org/task/23067 . Add

reboot=a

to the kernel command line in the bootloader. If it does not work try

reboot=a,w

.

Quote

6. Note: My primary harddisk OCZ Vertex 3 SSD does not have UEFI prefix in Boot menu. Only USB sticks obtain such prefix provided they contain FAT32 partition.

You should have a <EFI_SYSTEM_PARTITION>/efi/boot/bootx64.efi file for your SSD to have UEFI prefix in the boot menu.

Quote

7. Note: I didn't manage to run cdboot.efi from Windows DVD. When I booted into EFI shell, DVD wasn't mapped to some drive name like fs1: or something. I had to manually do "map fs1 blkA" - blkA was printed in the output of map command in the EFI shell. This mapping, however, didn't help: I changed drive to fs1 but 'ls' gave me an error. I coudn't browse the Windows DVD filesystem.

Most of the UEFI firmwares do not have support for reading ISO9660 or UDF iso/cd/dvd .

Quote

Info:
lshw output:
http://paste.pocoo.org/show/394135/

EFI shell commands output:
map.txt - http://paste.pocoo.org/show/394162/
pci.txt - http://paste.pocoo.org/show/394163/
ver.txt - http://paste.pocoo.org/show/394165/

EDIT: I duplicated post here: http://www.insanelym...p;#entry1687860
EDIT: I've just found this: https://help.ubuntu....i... as default. This seems to be a solution for the problem #2. I will check it by evening at home.

Actually it is possible to have Windows 7x64 ISO UEFI, Archboot ISO BIOS (using syslinux) and Archboot ISO UEFI in the same USB. I have done it before.

Just extract Windows 7 x64 iso and Archboot iso contents to the FAT32 formatted USB drive. Then extract the <WINDOWS_7_ISO>/efi/microsoft/boot/efisys.bin file using p7zip, it will contain <EFISYS.BIN>/efi/boot/bootx64.efi . Copy this bootx64.efi file to <USB>/efi/Microsoft/boot/bootmgfw.efi .
Similarly extract <ARCHBOOT_ISO>/efi/grub2/grub2_efi.bin and copy <GRUB2_EFI.BIN>/efi/boot/bootx64.efi to <USB>/efi/grub2/grub.efi . Copy the UEFI Shell to <USB>/efi/boot/bootx64.efi .

PS: I posted the same in insanelymac forums just in case. I am surprised though, you didn't read UEFI info available in Archwiki (UEFI and GRUB2 pages) and instead searched everything using Google.

[Keshav Padram]

@migle: I managed to make DUET boot into windows directly, without efivar.bin support and without the blue screen (using EmuVariableRuntimeDXE). This was made possible by http://tianocore.git...242d5b39cdc040a .

Just copy (ESP)/EFI/Microsoft/Boot/bootmgfw.efi to (ESP)/EFI/boot/bootx64.efi and the firmware will add a boot option in the firmware. If you allow DUET to try booting the available boot options in the Boot Manager (5 seconds timeout), after actual CD/DVD drive, the (ESP)/EFI/boot/bootx64.efi will be the second boot option and it will boot automatically. I guess this allows booting Windows UEFI-GPT via DUET without interruption. For this download the latest Efildr20 DUET file ( git commit 27-May-2011 ).

[Laqk]

You guys are lucky to be able to boot your Windows in EFI mode from your hard drive. I've been trying for days now without success. Migle's bootduet just doesn't work for me.

First, let me say I used Keshav's package to build an MBR DUET USB stick, and it works perfectly. I've used it to install Windows 7 x64 several times now on my GPT formatted hard disk. But so far I've failed in setting it up so it can boot without the USB stick.

Whatever I try I get the same result: I get the message "GPT Started!" on screen, then a blinking cursor in the upper left corner of the screen, then nothing. The problem is not with gpt.com, I confirmed that it does load and execute the boot sector of the EFI system partition by making the boot sector invalid on purpose (I compiled bootduet with -DWITH_VALIDATION and wrote zeros for the hidden sectors value) and I did get the "Bad" message, so I know the boot sector load and at least starts to execute. But when I compile bootduet with the -DDEBUG option I don't even get any number display on the screen, and I know that EFILDR20 is not only not loaded, it's not even searched for because even when the EFI system partition is completely empty, I don't get any "Missing EFILDR20" message.

I've created my GPT disk in several ways (let Windows create it itself during setup starting from a completely blank and unformated disk, created the GPT partition in Mac OS X, created it in Linux using gparted, and in this last case, letting windows create the EFI system partition itself, or creating it manually before windows setup using gparted and gnome disk manager). I tried a FAT32 EFI system partition using bd32hd.bin, and also a FAT16 one using bd16hd.bin, to no avail. Whatever I do I get the same result: the boot sector loads and starts executing, but somehow crashes along the way, doesn't even get to the debug code (when -DDEBUG is specified) and doesn't even try to load EFILDR (no "missing EFILDRXX" message when the file is not even on the disk).

In desperation, I began to suspect that maybe there was something wrong with my internal hard drive, so I took an 8 GiB USB stick and used gparted and gnome disk manager to format it in GPT, create two FAT32 4 GiB partition, and set the first one as an EFI system partition. Then I tried installing gpt.com on the master boot record and bootduet on the boot sector and tried booting with this USB stick, and got exactly the same result as always: "GPT Started!" message, and then the dreaded blinking cursor, then nothing.

I've tried every test, every configuration I could think of, but now I'm completely out of ideas. Maybe one of you guys could steer me in a new diection of investigation, or else I guess I'll just have to accept the fact that I'll have no choice but booting my machine with a USB stick...

[Keshav Padram]

Laqk, on May 30 2011, 07:30 AM, said:

Whatever I try I get the same result: I get the message "GPT Started!" on screen, then a blinking cursor in the upper left corner of the screen, then nothing. The problem is not with gpt.com, I confirmed that it does load and execute the boot sector of the EFI system partition by making the boot sector invalid on purpose (I compiled bootduet with -DWITH_VALIDATION and wrote zeros for the hidden sectors value) and I did get the "Bad" message, so I know the boot sector load and at least starts to execute. But when I compile bootduet with the -DDEBUG option I don't even get any number display on the screen, and I know that EFILDR20 is not only not loaded, it's not even searched for because even when the EFI system partition is completely empty, I don't get any "Missing EFILDR20" message.

The problem is with gpt.com in not passing the partition info to BootDuet bootsector. Gpt.com chainloads BootDuet fine but without the partition info (partition number or starting sector etc), BootDuet will fail to access the ppartition and load Efildr20. BTW did you try the memdisk method? Or instead of using gpt.com use some normal bios bootloader with gpt support like grub2 or syslinux in the MBR and then chainload the EFI System Partition from that. I currently use syslinux+BootDuet to boot into Windows 7 and linux (both direct from syslinux and via DUET using grub2-efi).

[Laqk]

Thanks for your reply Keshav. But I'm not sure I understand what you mean when you say that gpt.com doesn't pass partition info properly. If I read bootduet's source code correctly, all that it needs is the BIOS drive number of the boot drive, and the LBA of the boot sector of the EFI partition, and the "hd" versions of bootduet seem to read both of these directly from to boot sector itself (and I've made sure this information is present and accurate in the boot sector of the EFI partition). So it shouldn't need any information from gpt.com.

As for syslinux, I'll look into it. Besides syslinux and grub2, is there any other gpt-aware boot loader ?





@migle:

I think I found a discrepency in your bootduet source code.

In the comment, you say:

* 0x006000 - 0x007000 Room for several FAT sectors (4k) (FAT12)
...

* 0x007e00 - 0x008000 Room for one FAT sector (FAT32 and FAT16)


Yet, in the code, the following:

#if FAT == 32 || FAT == 16
.equ FatOffset, 0x6000 # Offset address for the FAT buffer
#else
.equ FatOffset, 0x7e00 # Offset address for the FAT buffer
#endif


seems to do the exact opposite. Is it a mistake in the comments ?

[migle]

Laqk, on May 31 2011, 03:23 AM, said:

Thanks for your reply Keshav. But I'm not sure I understand what you mean when you say that gpt.com doesn't pass partition info properly. If I read bootduet's source code correctly, all that it needs is the BIOS drive number of the boot drive, and the LBA of the boot sector of the EFI partition, and the "hd" versions of bootduet seem to read both of these directly from to boot sector itself (and I've made sure this information is present and accurate in the boot sector of the EFI partition). So it shouldn't need any information from gpt.com.

As for syslinux, I'll look into it. Besides syslinux and grub2, is there any other gpt-aware boot loader ?





@migle:

I think I found a discrepency in your bootduet source code.

In the comment, you say:

* 0x006000 - 0x007000 Room for several FAT sectors (4k) (FAT12)
...

* 0x007e00 - 0x008000 Room for one FAT sector (FAT32 and FAT16)


Yet, in the code, the following:

#if FAT == 32 || FAT == 16
.equ FatOffset, 0x6000 # Offset address for the FAT buffer
#else
.equ FatOffset, 0x7e00 # Offset address for the FAT buffer
#endif


seems to do the exact opposite. Is it a mistake in the comments ?

You are right, that is a bug. It is only of consequence for the FAT12 version.
The thing is there is no room for more than one sector at 7e00, but there is room for plenty at 6000.
Only the FAT12 code needs room for more than one, specifically for 3, because 3*sector-size is the lowest common multiple of the size of each FAT12 pointer (3 bytes) and the sector size, and that's the only way to keep the code simple.

That is not the bug which is causing you trouble.

The bug which is causing you trouble was one I introduced in commit 60cac65, one with a message like "Saved 5 bytes".
Apparently, neither Keshav nor nnobody ran into it, or if they did, they didn't tell me. They are using some previous version.

I haven't been using this (yet), so I didn't see it myself. It was a very stupid thing in the read function.
If you look at it, you'll see I push si, then the whole DAP structure, and after the int 13h, I only restore the si register, I forgot to skip those 16 bytes on top of the stack.

You see, I was eager to save those 5 bytes, and only truly saved 2.
I was so careful looking at the code listing, that I stupidly felt confident to commit the result without testing.

Sorry for the inconvenience.
Also, I didn't see your message earlier because (I just found out) I'm not subscribed to the entire topic, only to the threads I replied to (I think...).

Both the bug you pointed out and the very stupid and serious bug I just mentioned are now corrected in the repository.

...
ah!
and I took the time to test it this time...

Laqk, on May 30 2011, 03:00 AM, said:

You guys are lucky to be able to boot your Windows in EFI mode from your hard drive. I've been trying for days now without success. Migle's bootduet just doesn't work for me.

...

You know, I was wondering myself why was bootduet hanging with only a blinking cursor when I tried to run it once or twice....
But with so much updates on my laptop (I use Gentoo) I just overlooked and left it for latter.


I'll try to subscribe to the entire topic this time. If I had known before... this was a very easy fix.

[migle]

Keshav, on Apr 29 2011, 01:42 PM, said:

Duet does not have iso9660 fs driver. It has only fat12/16/32 driver. For bootx64.efi, extract /efi/microsoft/boot/efisys_noprompt.bin , and copy the /EFI/BOOT/BOOTX64.efi from that dir. Best method would be to simply extract the iso to a FAT32 usb pendrive and extract this file and set it up inside that pendrive. You can boot this pendrive instead of using iso, (which is slower)

I still don't have Windows running (it's also a new record, being able to do without Windows for a few months).

I'm trying to install Windows directly to the GPT partition using EFI again, because my installation media doesn't seem to have the repair option which is needed to generate the BCD.

I'm trying your suggestion now, copying everything to a USB pen. I formatted the pen in FAT32, installed BootDuet and EFILDR20 in it, copied the entire CD contents, extracted BOOTX64.EFI to EFI\Boot, all on the pen drive.
I tried two ways: having a single partition on the pen drive and using no partitions: using the entire drive for the FAT32 file system.

When DUET boots, I use "Boot from file" to try to boot BOOTX64.EFI, however it never seems to work. I always return to the previous screen.

Did you actually get this to work?
What did I do wrong?

Miguel

[Laqk]

Works perfectly now, thank you !

migle, on Jun 5 2011, 04:08 PM, said:

You see, I was eager to save those 5 bytes, and only truly saved 2.
I was so careful looking at the code listing, that I stupidly felt confident to commit the result without testing.

  Don't be too hard on yourself, It's just the "But I made only one minor and obvious modification, what could possibly go wrong ?" syndrome. I was plagued with it extensively in my programming days.

By the way, did you know that if you removed the check on the hiddensectors value (either in the code or by removing -DWITH_VALIDATION in the makefile), you can use your bootduet on an unpartitionned usb "superfloppy" (by placing it directly on sector 0) and it works splendidly ? Much simpler that way.

And as a small token of my appreciation, I've solved your EFI Windows 7 USB stick issue. Do the following:

1. Format your USB stick using the FAT32 format (an unpartitionned "superfloppy" works fine);

2. Install Bootduet on it;

3. Copy EFILDR20 to the root;

4. Copy the entire content of the Windows 7 x64 install DVD to the USB stick;

5. In folder EFI, create a folder named BOOT;

6. In that folder, copy the BOOTX64.EFI file that I included in this post.

And voila ! A perfectly functional EFI Windows 7 installation USB stick. I tested it and it works. I read your previous post and the problem was that the BOOTX64.EFI file included in the EFISYS.BIN and EFISYS_NOPROMPT.BIN don't work. They're only for the el-torito partition of a bootable DVD. The one I included in my post comes directly from the EFI system partition of an already installed and working Windows 7 x64 installation on an EFI GPT disk.

Enjoy ! And thanks again for your help.

 bootx64.zip   656.99K   27 downloads

[migle]

Laqk, on Jun 7 2011, 12:38 AM, said:

By the way, did you know that if you removed the check on the hiddensectors value (either in the code or by removing -DWITH_VALIDATION in the makefile), you can use your bootduet on an unpartitionned usb "superfloppy" (by placing it directly on sector 0) and it works splendidly ? Much simpler that way.

Of course I know! ...
I also tried the superfloppy thing when trying to boot the installation.

I'm ready to remove the HiddenSectors != 0 check if there is consensus.
That check however tests for the most likely installation error: the user not reading the TXT file and forgetting to set the HiddenSectors.

I know, people are having other problems instead of those I anticipated: nnobody found that it didn't work with Gpt.S and you ran into that bug, however...

Laqk, on Jun 7 2011, 12:38 AM, said:

And as a small token of my appreciation, I've solved your EFI Windows 7 USB stick issue. Do the following:

1. Format your USB stick using the FAT32 format (an unpartitionned "superfloppy" works fine);

2. Install Bootduet on it;

3. Copy EFILDR20 to the root;

4. Copy the entire content of the Windows 7 x64 install DVD to the USB stick;

5. In folder EFI, create a folder named BOOT;

6. In that folder, copy the BOOTX64.EFI file that I included in this post.

And voila ! A perfectly functional EFI Windows 7 installation USB stick. I tested it and it works. I read your previous post and the problem was that the BOOTX64.EFI file included in the EFISYS.BIN and EFISYS_NOPROMPT.BIN don't work. They're only for the el-torito partition of a bootable DVD. The one I included in my post comes directly from the EFI system partition of an already installed and working Windows 7 x64 installation on an EFI GPT disk.

Enjoy ! And thanks again for your help.

bootx64.zip

Well, thanks. It will be a clue, if nothing else. It's Vista 64 I'm trying to install. That's what came with my laptop, and I'm not interested in having the latest and greatest version...
Your explanation is what I expected. Maybe that BOOTX64 works with Vista install, if not, I'll get the BOOTX64 from a Vista installation (or install disk... it will be there).

The more long term solution is what Keshav said: using the iso 9660 driver from Virtual Box. I'm trying to avoid the trouble of understanding that complicated build process.

[Keshav Padram]

I checked bootx64.efi from efisys.bin and bootmgfw.efi from my Windows installation and they were indeed different. It is possible to take the bootx64.efi (actually bootmgfw.efi) file attached by laqk from "C:\Windows\Boot\EFI" from a Windows x64 installation or from the "install.wim" file in the "sources" folder of the Windows x64 ISO (Use 7-zip to extract the wim file).

[Keshav Padram]

@balta2ar: https://bbs.archlinu...=944722#p944722

[srs5694]

Hi all,

I finally got around to doing some serious testing with UEFI DUET. I'd tried it several months ago, and also several months before that, with limited success -- it hung most of my test machines at the time, and was impractical with others because of weird keyboard problems.

This time I've had more luck, and the new BootDuet helps a lot. I quickly got frustrated with all the manual fiddling when I wanted to re-install or try a different version, though. I therefore put together a Linux Bash script that does all the tricky stuff. I'll give it its own formal Web page eventually, but for now, here's a direct download link (edited for latest version -- 6/14/2011):

http://www.rodsbooks...stall-0.1.2.tgz

Launch the script with a -h or --help option to see a help summary. For convenience, you may want to edit the paths to the SYSLINUX, BootDuet, and UEFI DUET directories.

What it does, in summary:

• Optionally creates a FAT filesystem on the target partition (hard disk or USB flash drive) and changes the partition type code to that for an EFI System Partition when creating a filesystem.
  
• Adjusts the "hidden sectors" value in the FAT filesystem.
  
• Sets the "BIOS bootable" GPT flag or the "boot/active" MBR flag, as appropriate.
  
• Optionally installs SYSLINUX (for GPT or MBR) to the MBR's boot code area.
  
• Installs BootDuet to the specified partition's boot sector. It installs the LBA-32 version by default, but the LBA-64 version is an option.
  
• Installs the Efildr file from Keshav's UEFI DUET package. It installs the Efildr20 file from the UEFI 2.3 directory by default, but it will install the 2.1 version or 2.3's Efildr*_FSVariable version on request.
  
• Copies various .efi support files into an EFI directory on the target partition.

In summary, if you start with a disk with a FAT partition you want to preserve or any type of partition you're willing to sacrifice, you should get a bootable UEFI DUET disk out of it, although it will only boot to UEFI DUET.

What it does not do, in summary:

• Does not compile software. At a minimum, you'll need to compile BootDuet yourself. You may also need to install and perhaps compile SYSLINUX if you want to install it.
  
• Does not back up existing MBR or boot sector code. (That's on my to-do list.)
  
• Does not create or delete partitions.
  
• Does not clear other partitions' "BIOS bootable" or "boot/active" flags.
  
• Does not use the Gpt.com or Mbr.com MBR boot loader from Keshav's UEFI DUET distribution.
  
• Does not support the "hardcoded drive" method of BootDuet installation.
  
• Does not support UEFI variants other than Keshav's UEFI DUET. The issue is mainly one of locating files; the script assumes files are in certain locations. Of course, you could copy the Efildr file and supporting .efi files manually.
  
• Does not install any EFI boot loader other than the UEFI DUET software -- if you want ELILO, GRUB 2, rEFIt, etc., you'll need to install them yourself. Fortunately, that's fairly easy -- just copy some files.

Speaking of boot loaders, rEFIt is compatible with UEFI; however, you need a 64-bit-only binary, at least with UEFI DUET. The most common rEFIt binaries are 32/64-bit. I've successfully used the binaries that ship with Ubuntu with UEFI DUET, the UEFI in VirtualBox, and an Intel board with UEFI booting.

So far I've tested my duet-install script on USB flash drives. It certainly makes testing with UEFI DUET easier. I haven't yet gotten around to testing it on hard disks.

Keshav and migle, you're both welcome to include this script in your packages. IMHO, a single binary package with everything needed to install UEFI DUET on a hard disk would help make it a viable tool for people who need it. Unfortunately, my script makes heavy use of Linux-specific programs and features, so it's not likely to be very portable. (I'm not even sure how portable it is across Linux distributions -- so far I've tested it only on Gentoo.)

Naturally, I welcome any comments you might have.

[Keshav Padram]

srs5694, on Jun 12 2011, 12:47 AM, said:

• Installs the Efildr file from Keshav's UEFI DUET package. It installs the Efildr20 file from the UEFI 2.3 directory by default, but it will install the 2.1 version or 2.3's Efildr*_FSVariable version on request.
  
• Copies various .efi support files into an EFI directory on the target partition.

I actually renamed the EDK2_X64 to UDK_X64 to follow upstream naming (http://sourceforge.n...p?title=Welcome). I suggest people use the copy_duet_files.sh script in EFI_DUET dir instead of copying the files in duet-install.sh . You can also call the copy_duet_files script within duet-install.sh .

Quote

• Does not support UEFI variants other than Keshav's UEFI DUET. The issue is mainly one of locating files; the script assumes files are in certain locations. Of course, you could copy the Efildr file and supporting .efi files manually.

Same as above.

Quote

Keshav and migle, you're both welcome to include this script in your packages. IMHO, a single binary package with everything needed to install UEFI DUET on a hard disk would help make it a viable tool for people who need it. Unfortunately, my script makes heavy use of Linux-specific programs and features, so it's not likely to be very portable. (I'm not even sure how portable it is across Linux distributions -- so far I've tested it only on Gentoo.)

Naturally, I welcome any comments you might have.

1) Is it possible for you to host this script in a git repo somewhere with version history?

2) Please do not use UEFI 2.1 and UEFI 2.3 as options for Efildr. Use EDK_UEFI64 and UDK_X64 .

EDK_UEFI64 FSVariable works properly but is slow in booting and does not support USB Keyboards and AHCI. It causes no Blue screen in Windows x64. This happens to support Spec version at 2.1 .

EDK2_X64 supports AHCI, USB Keyboards (I modified the Makefile to include USB KB module) and is currently at UEFI Spec 2.31 (it usually matches the latest spec version at http://www.uefi.org/specs which is not necessarily only 2.3). With FSVariable it causes Windows Blue Screen but without FSVariable (using EmuVarRuntimeDxe instead) it boot successfully. I suggest removing copying FSVariable option in duet-install since i included them mainly to help migle test its the FSVariable problem. I suggest copying the FSVariable UDK_X64 Efildr20 manually if required.

3) Many stuff like fatBitness=`file -s $targetPart | sed 's/FAT (/\nFAT /' | grep bit | sed 's/)/\n/' | grep bit | sed 's/FAT //' | sed 's/ bit//'` (line 145)

can be done by

keshav_pr@my-comp ~ % blkid -p -i -o udev /dev/sda1
ID_FS_LABEL=EFISYS
ID_FS_LABEL_ENC=EFISYS
ID_FS_UUID=1BF5-7E28
ID_FS_UUID_ENC=1BF5-7E28
ID_FS_VERSION=FAT32
ID_FS_TYPE=vfat
ID_FS_USAGE=filesystem
ID_IOLIMIT_MINIMUM_IO_SIZE=512
ID_IOLIMIT_PHYSICAL_SECTOR_SIZE=512
ID_IOLIMIT_LOGICAL_SECTOR_SIZE=512
ID_PART_ENTRY_SCHEME=gpt
ID_PART_ENTRY_NAME=EFI_SYSTEM_PART
ID_PART_ENTRY_UUID=ed5cd10c-939a-49c1-ae67-b7f2a76949dd
ID_PART_ENTRY_TYPE=c12a7328-f81f-11d2-ba4b-00a0c93ec93b
ID_PART_ENTRY_NUMBER=1
ID_PART_ENTRY_OFFSET=2048
ID_PART_ENTRY_SIZE=819200
ID_PART_ENTRY_DISK=8:0

This is util-linux blkid, not e2fsprogs one. See http://git.kernel.or...0e25f0d8923d302 .

I am using udev-171 and util-linux-2.19 .

Also

   # Find partition number & whole-disk device node
   partNum=`udevadm info -q property -n $targetPart | grep UDISKS_PARTITION_NUMBER | cut -f 2 -d "="`
   echo "Partition number is $partNum"
   sysfsPath=`udevadm info -q property -n $targetPart | grep UDISKS_PARTITION_SLAVE | cut -f 2 -d "="`
   targetDiskNode=`cat $sysfsPath/uevent | grep DEVNAME | cut -f 2 -d "="`
   targetDisk="/dev/$targetDiskNode"
   echo "Target disk (for storing MBR boot code) is $targetDisk"

   # Find offset to partition
   partStartSector=`cat $sysfsPath/$targetDiskNode$partNum/start`

udevadm info -q property -n $targetPart | grep UDISKS returns only

keshav_pr@my-comp ~ % udevadm info -q property -n /dev/sda | grep -i UDISKS
UDISKS_PRESENTATION_NOPOLICY=0
UDISKS_ATA_SMART_IS_AVAILABLE=1
keshav_pr@my-comp ~ % udevadm info -q property -n /dev/sda1 | grep -i UDISKS
UDISKS_PRESENTATION_NOPOLICY=0

So obviously there is no UDISKS_PARTITION_SLAVE etc.

Also I only host Efildr20 (FAT32 specific file) in EFI_DUET. How will it work with BootDuet in FAT12/FAT16 even if you rename the files to EFILDR or EFILDR16 respectively?

[srs5694]

Keshav, on Jun 12 2011, 08:07 AM, said:

I actually renamed the EDK2_X64 to UDK_X64 to follow upstream naming (http://sourceforge.n...p?title=Welcome). I suggest people use the copy_duet_files.sh script in EFI_DUET dir instead of copying the files in duet-install.sh . You can also call the copy_duet_files script within duet-install.sh .

OK. I was working from a 6-day-old version that didn't include the copy_duet_files.sh script. One issue is that I see no way to specify the *_FSVariable variants of Efildr20 with that script. Since you say later that this version has limited appeal, though, it might not be a great loss.

Quote

1) Is it possible for you to host this script in a git repo somewhere with version history?

Possible? Certainly. Will I do it? I'm not sure. For me, this was a relatively quick hack, and I'm not sure I want to put a lot of effort into maintaining it over time.

Quote

2) Please do not use UEFI 2.1 and UEFI 2.3 as options for Efildr. Use EDK_UEFI64 and UDK_X64 .

Why? The problem with those names is that they're completely unclear as to which one is more recent. Somebody who's as immersed in UEFI DUET as you are no doubt knows the difference, but for everybody else, "2.1" and "2.3" are much more intuitive.

Quote

3) Many stuff like fatBitness=`file -s $targetPart | sed 's/FAT (/\nFAT /' | grep bit | sed 's/)/\n/' | grep bit | sed 's/FAT //' | sed 's/ bit//'` (line 145)

can be done by

[code]keshav_pr@my-comp ~ % blkid -p -i -o udev /dev/sda1

That might be a bit neater and possibly more reliable. I'll look into it.

Quote

This is util-linux blkid, not e2fsprogs one.

That could be an issue, though, if the e2fsprogs version is likely to be installed in some common environment.

Quote

udevadm info -q property -n $targetPart | grep UDISKS returns only

I was afraid the udevadm stuff might break on some distributions. I was using it in an effort to avoid assumptions about device names, but maybe it would be better to do that -- assume a device name in the form /dev/[sh]d?##, and work from that....

Quote

Also I only host Efildr20 (FAT32 specific file) in EFI_DUET. How will it work with BootDuet in FAT12/FAT16 even if you rename the files to EFILDR or EFILDR16 respectively?

It's worked fine in my tests, although I've only tried FAT12 a couple of times. FAT16 is much more common because I've been testing with 100-200MiB partitions (typical sizes for EFI System Partitions), and mkdosfs uses FAT16 on partitions of that size by default.

What is it about those files that's filesystem-specific? That is, is there something about how they begin to load themselves that makes assumptions about the filesystem, or do they assume they're running from a particular filesystem after they've been booted, or what? If you think problems are likely to result, even on some systems, from using a FAT32-specific build on a FAT16 filesystem, I recommend building both (and FAT12, too, for completeness), since it's hard to guarantee that the target filesystem will use a particular FAT level.

[Keshav Padram]

srs5694, on Jun 12 2011, 11:17 PM, said:

OK. I was working from a 6-day-old version that didn't include the copy_duet_files.sh script. One issue is that I see no way to specify the *_FSVariable variants of Efildr20 with that script. Since you say later that this version has limited appeal, though, it might not be a great loss.

Actually I made those changes locally in my system and pushed it only yesterday.

Quote

Possible? Certainly. Will I do it? I'm not sure. For me, this was a relatively quick hack, and I'm not sure I want to put a lot of effort into maintaining it over time.

Its just that I prefer to have a version history, both for viewing how the script has evolved and to revert any changes made whenever necessary.

Quote

Why? The problem with those names is that they're completely unclear as to which one is more recent. Somebody who's as immersed in UEFI DUET as you are no doubt knows the difference, but for everybody else, "2.1" and "2.3" are much more intuitive.

The 2.3 won't stay 2.3 forever. But its your call. AFAIK there is no noticable difference between the 2.1 and 2.3 firmwares except when AHCI and USB keyboards are involved.

Quote

That could be an issue, though, if the e2fsprogs version is likely to be installed in some common environment.

The e2fsprogs blkid has been deprecated by e2fsprogs devs a long time back. Although I use Archlinux which is rolling release with always the recent version in the repos, I don't think even Debian users will need to bother about blkid.

Quote

It's worked fine in my tests, although I've only tried FAT12 a couple of times. FAT16 is much more common because I've been testing with 100-200MiB partitions (typical sizes for EFI System Partitions), and mkdosfs uses FAT16 on partitions of that size by default.

You can explicitely specify mkfs.vfat -F32 or -F16 to define the FAT bitness.

Quote

What is it about those files that's filesystem-specific? That is, is there something about how they begin to load themselves that makes assumptions about the filesystem, or do they assume they're running from a particular filesystem after they've been booted, or what? If you think problems are likely to result, even on some systems, from using a FAT32-specific build on a FAT16 filesystem, I recommend building both (and FAT12, too, for completeness), since it's hard to guarantee that the target filesystem will use a particular FAT level.

The EDK and EDK2 build system create 3 firmware files Efildr (FAT12 - mainly for floppy or the memdisk image), Efildr16 (FAT16) and Elildr20 (FAT32). I copy only the Efildr20 compiled file to the EFI_DUET git repo. This link https://github.com/m...b/master/README explains clearly how the three files are setup. I suppose when it comes to BootDuet, any file can work with any FAT fs (haven't tested this myself).

[srs5694]

Keshav, on Jun 12 2011, 02:16 PM, said:

Actually I made those changes locally in my system and pushed it only yesterday.

I've uploaded a new version and updated the link above; but to repeat it so you don't need to find it, it's:

http://www.rodsbooks...stall-0.1.1.tgz

Regarding your installation script, I've modified mine to use it, but I've discovered one issue: Because you use the sudo command in the script, it won't work when run from PartedMagic, which doesn't include sudo. It'd be nice to be able to run it from PartedMagic, so as to prepare a hard disk before installing an OS on it.

Quote

The 2.3 won't stay 2.3 forever. But its your call. AFAIK there is no noticable difference between the 2.1 and 2.3 firmwares except when AHCI and USB keyboards are involved.

For me, 2.3 doesn't work at all on one of my computers, but 2.1 does. (I wish it were the other way around, obviously!) 2.3 crashes with a page fault or illegal instruction or some such message.

Quote

The e2fsprogs blkid has been deprecated by e2fsprogs devs a long time back. Although I use Archlinux which is rolling release with always the recent version in the repos, I don't think even Debian users will need to bother about blkid.

The new script checks the blkid version and uses it if it's the util-linux version, but keeps the old code in case it's not.

FWIW, I've also changed the way it splits off the whole-disk device filename and partition number. This has enabled me to eliminate all use of udevadm, which should make the script more portable; but there's the drawback that the new method is built on assumptions about how the device nodes are named (where the split between the whole-disk device and partition number occurs, mainly). This should be OK on hard disks using conventional names, but not if somebody uses a peculiar name or symbolic link or tries to install to a RAID device.

Quote

You can explicitely specify mkfs.vfat -F32 or -F16 to define the FAT bitness.

To a point, yes; but:

• You can't create a FAT32 filesystem on a device that's smaller than 66,586 sectors (about 32.5 MiB). That's pretty tiny, of course, but if you wanted to keep the UEFI DUET files separate from the EFI System Partition for some reason, you might want a tiny partition for UEFI DUET alone.
  
• You might want to live with whatever you've got because it already exists and holds files.

FWIW, I've modified my script to use FAT32 whenever it creates a filesystem on a partition larger than 66,586 sectors and to let mkdosfs decide what to do for smaller partitions.

I'm unlikely to make significant additional changes for several days, at least; I've got too much else on my plate this week.

[Keshav Padram]

srs5694, on Jun 13 2011, 06:39 AM, said:

I've uploaded a new version and updated the link above; but to repeat it so you don't need to find it, it's:

http://www.rodsbooks...stall-0.1.1.tgz

Regarding your installation script, I've modified mine to use it, but I've discovered one issue: Because you use the sudo command in the script, it won't work when run from PartedMagic, which doesn't include sudo. It'd be nice to be able to run it from PartedMagic, so as to prepare a hard disk before installing an OS on it.

I removed sudo usage and added check for root.

Quote

For me, 2.3 doesn't work at all on one of my computers, but 2.1 does. (I wish it were the other way around, obviously!) 2.3 crashes with a page fault or illegal instruction or some such message.

Thats strange. Since the old "2.1" version was the one with which you had the weird keyboard problems.

I have uploaded a newer build of EFI_DUET with your duet-install 0.1.1 in the git repo. Please test it. Thanks for your script.

[srs5694]

I've just tested it, and the new version with my script works for me. It's necessary to manually specify the paths to BootDuet and UEFI DUET, though, or edit them appropriately. I just realized that you include BootDuet in UEFI DUET, so it's possible to do this on lines 22-23:

uefiDuetPath=~/bootduet/skodabenz-EFI_DUET-157945c
bootDuetPath=$uefiDuetPath/BootSector

In fact, if the user doesn't move the script from the skodabenz-EFI_DUET-{version} directory, those two lines can be changed to:

thisScript=$(readlink -f $0)
uefiDuetPath=`dirname $thisScript`
bootDuetPath=$uefiDuetPath/BootSector

Those changes then obviate the need to use the -b and -u options or edit the script to customize the locations. Using -s might still be necessary, although the default setting is where it goes on every distribution I've tried if you install the syslinux package.

FWIW, GitHub seems to strip the executable bit from scripts -- or was that your decision?

[Keshav Padram]

srs5694, on Jun 14 2011, 08:18 AM, said:

I've just tested it, and the new version with my script works for me. It's necessary to manually specify the paths to BootDuet and UEFI DUET, though, or edit them appropriately. I just realized that you include BootDuet in UEFI DUET, so it's possible to do this on lines 22-23:

uefiDuetPath=~/bootduet/skodabenz-EFI_DUET-157945c
bootDuetPath=$uefiDuetPath/BootSector

In fact, if the user doesn't move the script from the skodabenz-EFI_DUET-{version} directory, those two lines can be changed to:

thisScript=$(readlink -f $0)
uefiDuetPath=`dirname $thisScript`
bootDuetPath=$uefiDuetPath/BootSector

Those changes then obviate the need to use the -b and -u options or edit the script to customize the locations. Using -s might still be necessary, although the default setting is where it goes on every distribution I've tried if you install the syslinux package.

But please do not remove the -b and -u options in the script.

Quote

FWIW, GitHub seems to strip the executable bit from scripts -- or was that your decision?

I maintain the repo in a ntfs-3g mounted partition (with exec permissions) and filemode = false set in the git repo .config file. So maybe git push ignores the filemode or github does. I think its the first one.

[migle]

Laqk, on Jun 7 2011, 12:38 AM, said:

And as a small token of my appreciation, I've solved your EFI Windows 7 USB stick issue. Do the following:

1. Format your USB stick using the FAT32 format (an unpartitionned "superfloppy" works fine);

2. Install Bootduet on it;

3. Copy EFILDR20 to the root;

4. Copy the entire content of the Windows 7 x64 install DVD to the USB stick;

5. In folder EFI, create a folder named BOOT;

6. In that folder, copy the BOOTX64.EFI file that I included in this post.

And voila ! A perfectly functional EFI Windows 7 installation USB stick. I tested it and it works. I read your previous post and the problem was that the BOOTX64.EFI file included in the EFISYS.BIN and EFISYS_NOPROMPT.BIN don't work. They're only for the el-torito partition of a bootable DVD. The one I included in my post comes directly from the EFI system partition of an already installed and working Windows 7 x64 installation on an EFI GPT disk.

I was able to install Vista with the BOOTX64.EFI you sent from Windows 7. I didn't even bother to look for the corresponding file in the Vista DVD.

It also wasn't necessary to install Duet on the pen drive: I already had it on the hard disk, and because it looks for bootable removable media, it immediatly booted the BOOTX64.EFI file without even asking.

It didn't work the first time: I had to sort my partitions and create a Microsoft Reserved partition of 128MB (which I still don't know if it is actually in use). Then it worked.

Since then, I installed windows some four times: I'm ending up with a system which gives errors when I try to install some of the updates. However, I think that is related to the order by which I install the updates and the software for my laptop, and not related to using DUET. I'll try a fifth time one of these days.

Thanks,

srs5694, on Jun 11 2011, 08:17 PM, said:

Keshav and migle, you're both welcome to include this script in your packages. IMHO, a single binary package with everything needed to install UEFI DUET on a hard disk would help make it a viable tool for people who need it. Unfortunately, my script makes heavy use of Linux-specific programs and features, so it's not likely to be very portable. (I'm not even sure how portable it is across Linux distributions -- so far I've tested it only on Gentoo.)

Naturally, I welcome any comments you might have.

Greetings!
This is all good to hear. We're not far from turning DUET into something like EUET, Enthusiast's UEFI Emulation instead of Developer's :-)

I will only keep on github my source file. What I would like would be to completely tame the compilation process, which is too complicated for people that are not firmware developers, maybe taking advantage of those GNU EFI tools which are used to compile ELILO.

I think you're right by not changing partitions, boot flags, etc. That would only make new limitations arise. Partitioning is the job of gdisk and similar tools. The model would be GRUB. About compiling, I don't know... you use Gentoo, you know: picking things from their source repositories, patching, compiling, is pretty practical.

I'll comment about FAT bitness next.

srs5694, on Jun 12 2011, 06:47 PM, said:

What is it about those files that's filesystem-specific? That is, is there something about how they begin to load themselves that makes assumptions about the filesystem, or do they assume they're running from a particular filesystem after they've been booted, or what? If you think problems are likely to result, even on some systems, from using a FAT32-specific build on a FAT16 filesystem, I recommend building both (and FAT12, too, for completeness), since it's hard to guarantee that the target filesystem will use a particular FAT level.

My guess is that only the first 512 bytes of EFILDR, EFILDR16 and EFILDR20 is FAT bitness specific.
And because of that, and because that FAT loading code has all those limitations we detected, my BootDuet jumps over those first 512 bytes. I squeezed more general FAT code into the boot program than there was on DUET's bootsect.S and start32.S combined.
That's why I jump to 2000:0200 instead of 2000:0000.

After the entire flash image (the EFILDR file) is loaded, any access to a file system is for looking for boot programs, and that is using the FAT driver within the EFILDR image (which surely supports all three types of FAT).

This is my guess, because I never had the EFILDR and EFILDR16 images with me. Only the EFILDR20 that Keshav compiled (call it the law of the least effort, or the principle of minimum energy...).
I'm convinced that if you take the first 512 or maybe 4096 bytes of those three files, the rest of the files are byte for byte equal.

From the beginning I have done what you did: renamed EFILDR20 to EFILDR16 to use it on a FAT16 file system.

The ESP partition on my laptop is 16-bit too, because that's the dosfstools default for a 128MB filesystem, and it's the more logical for that size, because less space is consumed by the File Allocation Table, and it's also the safest, because it's troublesome to distinguish between FAT file system versions (there isn't a field in the BPB saying the bitness of the FAT), and some tools distinguish between FAT 12 and FAT 16 by number of sectors in the file system.

Maybe some people were put away from FAT12 and FAT16 by a bug I left in the code after making some changes. At this moment FAT16 surely works well.

I think the best approach is formating the filesystem using default parameters and then installing the correct version of BootDuet for it.
Even better: don't format the filesystem, let the user decide what files he keeps in there, just check the hidden sectors field. This is just as you let the user create the partitions, etc. Partitioning and formatting automatically only makes sense for the original use of DUET which was on a dedicated pen drive, you can never make those kinds of assumptions about someones personal harddisk.

Because we jump over the first 512 bytes of EFILDR, it also no longer makes sense to use a different file name for each FAT bitness. We should just use EFILDR...
When we completely take hold of the compilation process, we can also produce an image without those 512 bytes.

By the way, I also tried to make BootDuet support all three types of FAT bitness, but I just couldn't. I'd have to use only the 420 bytes available for a FAT32 boot program... I couldn't even squeeze the FAT12 version into 420 bytes.
So, the BootDuet image is the only thing that is FAT bitness specific.