Jump to content

Clover Problems and Solutions

ErmaC

By ErmaC,
in Clover Development

 Share
3,206 posts in this topic

Recommended Posts

Slice
Slice
Posted
Posted

i see 3 floating mtoc starting from page 27. Which one you recommend?

Tested all three versions I found a little difference.

Only my old version is compiled for MACOSXMINVERSION=10.6 and it is only working version in 10.6.8.

I think I will upload it to sf.net not excuding usage of custom version.

Anyway I see no differences in final result.

  • Like 1
Link to comment
Share on other sites
cecekpawon
cecekpawon
Posted
Posted

Would like to suggest adding "--clover-toolchain-dir=<TOOLCHAIN_DIR>" to set "$TOOLCHAIN_DIR" via args (edk like --edk-tools-path) in "ebuild.sh", so user can decide what version of required tool path to be use. Yes, we can use @STLVNUBs "extras.sh" to build that tools which can pass via args too about what version that user want for each: gcc, nasm, gettext, etc. Please do not make it (build script) more complicated, since @apianti appear to port it (very hardly) to make it run under Windows too (downloading VS2012 now after couple of tries build with gcc wihout success under Windows, which do not recommended by edk). What I can tell. Clover build should run well on Macos, Linux & Windows as advertise. What you people think?

  • Like 2
Link to comment
Share on other sites
Zenith432
Zenith432
Posted
Posted

left shift by 1 of a signed integer is identical to multiplication by 2, with exact same overflow conditions.

left shift by 1 of an unsigned integer is identical to multiplication by 2, with exact same overflow conditions.

right shift by 1 of a signed integer (= arithmetic shift) is identical to division by 2 with round toward zero.

right shift by 1 of an unsigned integer (= logical shift) is identical to division by 2 with round toward zero.

 

Why does a complier that does not warn about possible overflow of multiplication think it's necessary to warn about shifts?

 

And same compiler thinks it's perfectly okay to silently emit a trap into the code with no warning for ((int*) 0)[index].

  • Like 1
Link to comment
Share on other sites
smolderas
smolderas
Posted
Posted

Sorry if this is a known issue or my idiot thinking, with the latest revision (edk2@r20654 and Clover@r3480 and applied the Patches_for_EDK2 too) I have following issue:

I'm using following command:

./ebuild.sh -gcc49 -release --vbios-patch-cloverefi --only-sata0 -D USE_APPLE_HFSPLUS_DRIVER --x64

And it ends with not founding /usr/lib/system/libsystem_stats.dylib for x86_64. I have Xcode 7.3 too though.

 

Any help?

 

Here is the output of the build script:

$ ./ebuild.sh -gcc49 -release --vbios-patch-cloverefi --only-sata0 -D USE_APPLE_HFSPLUS_DRIVER --x64
TOOLCHAIN_DIR: /Users/XX/Documents/CloverGrowerPro/edk2/Clover/../../toolchain
Initializing workspace
Loading previous configuration from /Users/XX/Documents/CloverGrowerPro/edk2/Conf/BuildEnv.sh
WORKSPACE: /Users/XX/Documents/CloverGrowerPro/edk2
EDK_TOOLS_PATH: /Users/XX/Documents/CloverGrowerPro/edk2/BaseTools
CONF_PATH: /Users/XX/Documents/CloverGrowerPro/edk2/Conf
Building tools as they are not found
/Applications/Xcode.app/Contents/Developer/usr/bin/make -C Source/C
Attempting to detect ARCH from 'uname -m': x86_64
Detected ARCH of X64 using uname.
mkdir -p .
/Applications/Xcode.app/Contents/Developer/usr/bin/make -C Common
make[2]: Nothing to be done for `all'.
/Applications/Xcode.app/Contents/Developer/usr/bin/make -C GnuGenBootSector
gcc -Wno-deprecated-declarations -o ../bin/GnuGenBootSector  GnuGenBootSector.o -L../libs -lCommon
ld: file not found: /usr/lib/system/libsystem_stats.dylib for architecture x86_64
collect2: error: ld returned 1 exit status
make[2]: *** [../bin/GnuGenBootSector] Error 1
make[1]: *** [GnuGenBootSector] Error 2
make: *** [Source/C] Error 2

Edit:

I now realize that it could be from an old SDK (10.9), I'm on 10.11...

 

Edit2:

-xcode5 gives the same error.

Link to comment
Share on other sites
Zenith432
Zenith432
Posted
Posted

SIC ---> ...

@Zenith432, for your next commit:
line 112 in ebuild5.sh - just misspell catched - echo "uilding it"

 

 

cecekpawon:
your clover_build_info does not support

TOOLCHAIN_DIR: /usr
./ebuild.sh: line 63: sw_vers: command not found
Initializing workspace
Loading previous configuration from /media/Dev/edk2/Conf/BuildEnv.sh
WORKSPACE: /media/Dev/edk2
EDK_TOOLS_PATH: /media/Dev/edk2/BaseTools
CONF_PATH: /media/Dev/edk2/Conf

Running edk2 build for CloverX64 using the command:
build  -D DISABLE_USB_SUPPORT -D USE_BIOS_BLOCKIO -D USE_LOW_EBDA -p Clover/Clover.dsc  -a X64 -b RELEASE -t GCC53 -n 3 

Build environment: Linux-4.4.7-300.fc23.x86_64-x86_64-with-fedora-23-Twenty_Three
Link to comment
Share on other sites
cecekpawon
cecekpawon
Posted
Posted

It generate "BUILDINFOS_STR" in "Version.h"? Mine like this:

// Autogenerated Version.h
#define FIRMWARE_VERSION "2.31"
#define FIRMWARE_BUILDDATE "2016-04-27 19:31:05"
#define FIRMWARE_REVISION L"3479"
#define REVISION_STR "Clover revision: 3479"
#define BUILDINFOS_STR "Args: cbuild.bat -a X64 -t VS2012x86 -b RELEASE :: Command: build -p \"G:\\src\\edk2\\Clover\\Clover.dsc\" -DNO_GRUB_DRIVERS  -a X64 -t VS2012x86 -b RELEASE :: OS: 10.0.10240"

Need tweak for $OSVER under Linux, something like uname or what

"Build environment" looks more informative for me. Generated by: /src/edk2/BaseTools/Source/Python/build/build.py?

 

My cbuild looks ugly without @apianti

 

 

 

      for /f "tokens=2 delims=[]" %%x in ('ver') do set WINVER=%%x
      set WINVER=%WINVER:Version =%

      rem build clover
      if x"%DSCFILE%" == x"" goto buildall
      rem build specific dsc
      echo Building selected ...
      build -DNO_GRUB_DRIVERS  %*
      if errorlevel 1 goto failscript
      if not x"%CLEANING%" == x"" goto:eof
      goto postbuild

      :buildall
         if x"%BUILD_ARCH%" == x"X64" goto build64

         set "ARCH_ARGUMENT="
         if x"%BUILD_ARCH%" == x"" set "ARCH_ARGUMENT=-a IA32"

         echo Building Clover (IA32) ...
         build -p "%WORKSPACE%\Clover\Clover.dsc" -DNO_GRUB_DRIVERS %ARCH_ARGUMENT% %*
         if errorlevel 1 goto failscript

         if x"%BUILD_ARCH%" == x"IA32" (
            if not x"%CLEANING%" == x"" goto:eof
            goto postbuild
         )

      :build64
         set "ARCH_ARGUMENT="
         if x"%BUILD_ARCH%" == x"" set "ARCH_ARGUMENT=-a X64"

         echo Building Clover (X64) ...
         set "cmd=build -p "%WORKSPACE%\Clover\Clover.dsc" -DNO_GRUB_DRIVERS %ARCH_ARGUMENT% %*"
         set "clover_build_info=Args: %~nx0 %* :: Command: %cmd% :: OS: %WINVER%"
         rem echo %clover_build_info%
         set clover_build_info=%clover_build_info:\=\\%
         set clover_build_info=%clover_build_info:"=\"%
         echo #define BUILDINFOS_STR "%clover_build_info%">> Version.h
         %cmd%
         rem build -p "%WORKSPACE%\Clover\Clover.dsc" -DNO_GRUB_DRIVERS %ARCH_ARGUMENT% %*
         if errorlevel 1 goto failscript

         if not x"%CLEANING%" == x"" goto:eof
         goto postbuild

 

 

 

Attached build from Windows

Link to comment
Share on other sites
Zenith432
Zenith432
Posted
Posted

Need tweak for $OSVER under Linux, something like uname

uname -a

Linux localhost.localdomain 4.4.7-300.fc23.x86_64 #1 SMP Wed Apr 13 02:52:52 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux

Also have /etc/os-release

 

NAME=Fedora

VERSION="23 (Workstation Edition)"

ID=fedora

VERSION_ID=23

PRETTY_NAME="Fedora 23 (Workstation Edition)"

ANSI_COLOR="0;34"

CPE_NAME="cpe:/o:fedoraproject:fedora:23"

HOME_URL="https://fedoraproject.org/"

BUG_REPORT_URL="https://bugzilla.redhat.com/"

REDHAT_BUGZILLA_PRODUCT="Fedora"

REDHAT_BUGZILLA_PRODUCT_VERSION=23

REDHAT_SUPPORT_PRODUCT="Fedora"

REDHAT_SUPPORT_PRODUCT_VERSION=23

PRIVACY_POLICY_URL=https://fedoraproject.org/wiki/Legal:PrivacyPolicy

VARIANT="Workstation Edition"

VARIANT_ID=workstation

 

 

The Version.h generated

#define FIRMWARE_VERSION "2.31"

#define FIRMWARE_BUILDDATE "2016-04-27 14:40:27"

#define FIRMWARE_REVISION L"3483"

#define REVISION_STR "Clover revision: 3483"

#define BUILDINFOS_STR "Args: ./ebuild.sh -gcc53 | Command: build -D USE_LOW_EBDA -p Clover/Clover.dsc -a X64 -b RELEASE -t GCC53 -n 3 -gcc53"

Link to comment
Share on other sites
tluck
tluck
Posted
Posted

so I can build and use clover UEFI built with Xcode or GCC5.3 - cool.

 

note: for pure Xcode (-xcode5) build..

 

- export TOOLCHAIN_DIR=/Applications/Xcode.app/Contents/Developer/usr

- renamed edk2/Clover/../../toolchain to toolchain-gcc53 , to make sure it was not using any of these the 3rd party tools

- add mtoc.NEW to /usr/local/bin

- add the 2 cross-compiler files in $TOOLCHAIN_DIR/cross/bin files sym-linked to ../../bin/gcc

 

./ebuild.sh

 

some binaries (.efi) were slightly smaller and some are slightly larger - so other than size ... 

 

anyway - which compile/build mode is preferred or "better" going forward - using xcode5 or gcc53? 

Link to comment
Share on other sites
Zenith432
Zenith432
Posted
Posted

Why does a complier that does not warn about possible overflow of multiplication think it's necessary to warn about shifts?

This is a C99 perversion

 

 

4 The result of E1 << E2 is E1 left-shifted E2 bit positions; vacated bits are filled with

zeros. If E1 has an unsigned type, the value of the result is E1 * 2^E2, reduced modulo

one more than the maximum value representable in the result type. If E1 has a signed

type and nonnegative value, and E1 * 2^E2 is representable in the result type, then that is

the resulting value; otherwise, the behavior is undefined.

 

5 The result of E1 >> E2 is E1 right-shifted E2 bit positions. If E1 has an unsigned type

or if E1 has a signed type and a nonnegative value, the value of the result is the integral

part of the quotient of E1 / 2^E2. If E1 has a signed type and a negative value, the

resulting value is implementation-defined.

 

 

 

and this is C89

 

 

The result of E1 << E2 is E1 left-shifted E2 bit positions; vacated

bits are filled with zeros. If E1 has an unsigned type, the value of

the result is E1 multiplied by the quantity, 2 raised to the power E2,

reduced modulo ULONG_MAX+1 if E1 has type unsigned long, UINT_MAX+1

otherwise. (The constants ULONG_MAX and UINT_MAX are defined in the

header <limits.h> .)

 

The result of E1 >> E2 is E1 right-shifted E2 bit positions. If E1

has an unsigned type or if E1 has a signed type and a nonnegative

value, the value of the result is the integral part of the quotient of

E1 divided by the quantity, 2 raised to the power E2 . If E1 has a

signed type and a negative value, the resulting value is

implementation-defined.

 

 

 

Update:

C99 says signed integers may be represented using 2's complement, 1's complement or sign-magnitude.

C89 doesn't say much about representation of signed integer except that the range of non-negative signed integer must be at least as large as unsigned integers.

Link to comment
Share on other sites
cecekpawon
cecekpawon
Posted
Posted

@Zenith432 please pick one which you preferred sir and insert something like below

 

 

 

if [[ "$SYSNAME" == Linux ]]; then
  ##
else
  OSVER="$(sw_vers -productVersion | sed -e 's/\.0$//g')" 
fi

 

 

 

Hope someone can test it under Windows with modified "cbuild.bat"

 

 

 

@echo off

git clone https://github.com/tianocore/edk2 edk2
cd edk2
git clone https://github.com/tianocore/edk2-BaseTools-win32 BaseTools\Bin\Win32
svn co svn://svn.code.sf.net/p/cloverefiboot/code/ Clover
rem edksetup
cd Clover
rem cbuild clean
rem cbuild -t VS2012x86 -b RELEASE
rem cbuild -a IA32 -t VS2012x86 -b RELEASE
cbuild -a X64 -t VS2012x86 -b RELEASE

 

 

  • Like 1
Link to comment
Share on other sites
Slice
Slice
Posted
Posted

Hi Zenith432,

Start32H2.com binary is not working now. Is it commit 3446?

Compare, please, with working (?) one.

Start64H2.com.zip

I make it by replacing first 512 byte from old working boot file in the new boot file that is not working.

 

 

Good news. Clover-3490 compiled by XCODE5 toolset in 10.7.5 (!) with Xcode 4.4.1

Снимок экрана 2016-04-29 в 16.58.49.png

and it works!

 

 

Link to comment
Share on other sites
Zenith432
Zenith432
Posted
Posted

So, if any1 is interested in simplified build of mtoc,

  • download latest cctools from Apple opensource (877.5) and unpack.
  • In the top directory, create a makefile Clover_cctools.mak with following text

 

all:
	make LTO= EFITOOLS=efitools -C libstuff
	make -C efitools
	strip efitools/mtoc.NEW efitools/makerelocs.NEW

clean:
	make -C efitools clean
	make -C libstuff clean

  • run 'make -f Clover_cctools.mak'.  The product is in efitools/mtoc.NEW.
This mtoc is built without LTO support.  I don't think LTO is needed, because mtoc converts finished EFI binary to coff - a process which does not involve LTO.

 

Second, if anyone is interested in a nasm that does not have all the macho32/macho64 bugs we had, build one as follows

  • Download nasm-2.12.02rc1 source code from here.
  • Apply the following patch at the top directory

 

diff a/output/outmacho.c b/output/outmacho.c
--- a/output/outmacho.c
+++ b/output/outmacho.c
@@ -465,6 +465,8 @@ static int64_t add_reloc(struct section *sect, int32_t section,
 	    r->snum = raa_read(extsyms, section);
 	    if (reltype == RL_BRANCH)
 		r->type = X86_64_RELOC_BRANCH;
+	    else if (r->type == GENERIC_RELOC_VANILLA)
+		adjust = -sect->size;
 	} else {
 	    /* local */
 	    r->ext = 0;
@@ -1321,6 +1323,8 @@ static void macho_write_section (void)
 		l += sectstab[r->snum]->addr;
 		if (r->pcrel)
 		    l -= s->addr;
+	    } else if (r->pcrel && r->type == GENERIC_RELOC_VANILLA) {
+		l -= s->addr;
 	    }
 
 	    /* write new offset back */

  • run ./configure in top directory, then 'make', then 'make strip'.  products are ./nasm, and ./ndisasm - you can copy them to whereever you place nasm.
  • Like 1
Link to comment
Share on other sites
Zenith432
Zenith432
Posted
Posted

Hi Zenith432,

Start32H2.com binary is not working now. Is it commit 3446?

Compare, please, with working (?) one.

attachicon.gifStart64H2.com.zip

Your Start64H2.com and Start64H3.com are swapped. The file you uploaded as "Start64H2.com" is actually Start64H3.com in the repository - it shows the character '5', and is meant for boot6, low-ebda build.

 

Start64H.com -- boot6, std-ebda, char 6

Start64H2.com -- boot7, std-ebda, char B

Start64H3.com -- boot6, low-ebda, char 5

Start64H4.com -- boot7, low-ebda, char L

 

this is the way it's set in the repository.

 

PS

If you run 'make -f Clover.mak clean all' in BootSector, it will rebuild all the Starts, and then svn status, resulting binary files should be exactly identical to svn. You can see in Clover.mak rules what each Start64H is... then compare to rule for their selection in ebuild.sh and see that it fits.

 

BTW... GCC 6.1 was released 2 days ago :)

 

Edit: tried GCC 6.0 on Fedora (as -gcc53), and I had to add '-Wno-lto-type-mismatch'.  There are multiple PeCoffLib.h with different definitions of PE_COFF_LOADER_IMAGE_CONTEXT, and VBoxFsDxe also multiply defines VOLSTRUCTNAME and DNODESTRUCTNAME (intentionally).

Link to comment
Share on other sites
Slice
Slice
Posted
Posted

Thanks for the explanations and for build rules. I will update my old instructions.

But you didn't catch what I said.

The boot file rev >3428 is not working. See, for example, ticket 241. Other users messages.

I reproduced the issue on my computer #4 in sig. The boot6 (actually boot5 low-ebda) hangs before show "5" on the screen. Just black screen.

If I boot in QEMU then it started but USB will not work. Something wrong with interrupts.

As I have working boot I compared frist 512 bytes. They are from Start64H3.com

See in picture. The overview is enough to see the difference.

Screen Shot 2016-04-29 at 19.53.57.png

Left one is old good Start64H3 dated 2014.

Right one non-working svn version dated 19.04.2016.


It seems the key difference is rev 3444

-        .long SYS_CODE64_SEL  # selector 15:0
+        .word SYS_CODE64_SEL  # selector 15:0
Link to comment
Share on other sites
Zenith432
Zenith432
Posted
Posted (edited)

That is the proper layout of IDT64.  Compare CpuDxe/X64/CpuInterrupt.S (or .nasm).  It's int16, int16, int8, int8, int16, int32, int32. (total 128 bits.)

Is the one on the left the same one you atached in post 668?

 

It could be an alignment problem

 

In CpuInterrupt.S, GDT_BASE is aligned 16, IDT_BASE is aligned 8.

 

I think it should be other way, since in 64-bit, GDT entry is 8 bytes, IDT entry 16 bytes.

 

However, in st32_64.S, only alignment commands are align 2 (p2align 1).

 

That said, the IDT on the right (which is the bad one) is aligned on 16 byte boundary.

 

It could be that the table needs an extra 0 entry at the end.  Modify st32_64.S like this

 

diff a/BootSector/st32_64.S b/BootSector/st32_64.S
--- a/BootSector/st32_64.S
+++ b/BootSector/st32_64.S
@@ -1109,6 +1109,8 @@ IDT_BASE:
         .long 0               # offset 63:32
         .long 0               # 0 for reserved
 
+        .fill 16, 1, 0
+
 IDT_END: 
 
         .p2align 1
rebuild the Starts with 'make -f Clover.mak' in BootSector, and then try if they still reproduce the problem. Edited by Zenith432
Link to comment
Share on other sites
Slice
Slice
Posted
Posted

That is the proper layout of IDT64.  Compare CpuDxe/X64/CpuInterrupt.S (or .nasm).  It's int16, int16, int8, int8, int16, int32, int32. (total 128 bits.)

Is the one on the left the same one you atached in post 668?

 

It could be an alignment problem

 

In CpuInterrupt.S, GDT_BASE is aligned 16, IDT_BASE is aligned 8.

 

I think it should be other way, since in 64-bit, GDT entry is 8 bytes, IDT entry 16 bytes.

 

However, in st32_64.S, only alignment commands are align 2 (p2align 1).

 

That said, the IDT on the right (which is the bad one) is aligned on 16 byte boundary.

Yes, I read about IDT, it should be 16 byte.

As I see the problem is somewhere else. I am trying to disassemble this two files now.

Good codes

EB 24 45 0C 6D 0C 36 0C 34 0C 54 0C 20 0C 55 0C
6E 0C 73 0C 75 0C 70 0C 70 0C 6F 0C 72 0C 74 0C
65 0C 64 0C 21 0C 8C C8 8E D8 8E C0 8E D0 BC E0
0F 66 BB 00 00 00 00 66 8D 3E 9C 0B 66 B8 20 E8
00 00 66 B9 14 00 00 00 66 BA 50 41 4D 53 CD 15
72 0C 66 83 C7 14 66 83 FB 00 74 02 EB DE 66 8D
06 9C 0B 66 29 C7 66 89 3E 98 0B 66 31 DB 8C CB
66 C1 E3 04 67 66 8D 83 6C 03 00 00 66 A3 68 03
67 66 8D 83 C0 03 00 00 66 A3 B6 03 67 66 8D 93
98 0B 00 00 B8 01 24 CD 15 73 06 E4 92 0C 02 E6
92 66 BB 0F 00 00 00 66 B8 35 0E 00 00 66 B9 10
00 00 00 CD 10 FA 66 8D 06 E4 02 66 05 06 00 02
00 66 A3 E4 02 66 8D 06 20 03 66 05 06 00 02 00
66 A3 20 03 66 0F 01 16 66 03 0F 20 C0 0C 01 0F
22 C0 66 EA 00 00 00 00 10 00 66 B8 08 00 8E D8
8E C0 8E D0 0F 20 E0 66 0F BA E8 05 0F 22 E0 B8
00 80 08 00 0F 22 D8 B9 80 00 00 C0 0F 32 0F BA
E8 08 0F 30 0F 20 C0 0F BA E8 1F 0F 22 C0 67 EA
00 00 00 00 38 00 66 B8 30 00 8E D8 66 B8 18 00
8E C0 8E D0 8E D8 BD 00 00 40 00 BB 00 00 07 00
48 33 C0 66 B8 B4 03 48 05 00 00 02 00 0F 01 18
48 C7 C0 00 10 02 00 50 C3 B9 00 00 E7 ED E4 64
24 02 E0 F8 C3 90 47 00 00 00 00 00 00 00 00 00

Bad

EB 24 45 0C 6D 0C 36 0C 34 0C 54 0C 20 0C 55 0C
6E 0C 73 0C 75 0C 70 0C 70 0C 6F 0C 72 0C 74 0C
65 0C 64 0C 21 0C 8C C8 8E D8 8E C0 8E D0 BC E0
0F 66 BB 00 00 00 00 66 8D 3E 44 0B 66 B8 20 E8
00 00 66 B9 14 00 00 00 66 BA 50 41 4D 53 CD 15
72 0C 66 83 C7 14 66 83 FB 00 74 02 EB DE 66 8D
06 44 0B 66 29 C7 66 89 3E 40 0B 66 31 DB 8C CB
66 C1 E3 04 67 66 8D 83 6C 03 00 00 66 A3 68 03
67 66 8D 83 C0 03 00 00 66 A3 B6 03 67 66 8D 93
40 0B 00 00 B8 01 24 CD 15 73 06 E4 92 0C 02 E6
92 66 BB 0F 00 00 00 66 B8 35 0E 00 00 66 B9 10
00 00 00 CD 10 FA 66 8D 06 E4 02 66 05 06 00 02
00 66 A3 E4 02 66 8D 06 20 03 66 05 06 00 02 00
66 A3 20 03 66 0F 01 16 66 03 0F 20 C0 0C 01 0F
22 C0 66 EA 00 00 00 00 10 00 66 B8 08 00 8E D8
8E C0 8E D0 0F 20 E0 66 0F BA E8 05 0F 22 E0 B8
00 80 08 00 0F 22 D8 B9 80 00 00 C0 0F 32 0F BA
E8 08 0F 30 0F 20 C0 0F BA E8 1F 0F 22 C0 67 EA
00 00 00 00 38 00 66 B8 30 00 8E D8 66 B8 18 00
8E C0 8E D0 8E D8 BD 00 00 40 00 BB 00 00 07 00
48 33 C0 66 B8 B4 03 48 05 00 00 02 00 0F 01 18
48 C7 C0 00 10 02 00 50 C3 B9 00 00 E7 ED E4 64
24 02 E0 F8 C3 90 47 00 00 00 00 00 00 00 00 00

The difference for example at address 0x3A

Link to comment
Share on other sites
 Share
Go to topic listing
×
×
  • Create New...