Clover General discussion

[Jief_Machak]

1 hour ago, Jief_Machak said:

I am still not 100% convinced that the problem is this

Yes, was right : disabling optimization for Settings.cpp wasn't the right way to solve this.

So far, I'd sat that this was a problem of clang optimizing out default ctor even when they were needed !

Let's check.

 

[kushwavez]

@Jief_Machak

now it's okay

5417624 : built with XCODE8, booted

Edited April 16, 2021 by kushwavez

[eng_redaesm]

@Jief_Machak commit 5417624 built with xcode 12.4. booting fine here

[matxpa]

@Jief_Machak & @Slice

 

Hi

 

r5133 XCODE8 (master, commit 541762427) compiled boots and works fine on :

 

MacBookPro11,3 ⌘  Powered by Clover revision: 5133 (master, commit 541762427)
macOS 11.2.3 (20D91) Darwin 20.3.0
Xcode Version 12.4 (12D4e) Commande line 2384 and Apple clang version 12.0.0 (clang-1200.0.32.29)

 

Spoiler

Boot Log (part of.)

 

0:154  0:000  Now is 16.04.2021,  10:20:29 (GMT)
0:154  0:000  Starting Clover revision: 5133 (master, commit 541762427) on American Megatrends EFI
0:154  0:000  Build id: 20210416125551-5417624
0:154  0:000  Build with: [Args: -D NO_GRUB_DRIVERS_EMBEDDED -D LESS_DEBUG -t XCODE8 | -D NO_GRUB_DRIVERS_EMBEDDED -D LESS_DEBUG --conf=/Users/meunier/CloverBootloader/Conf -D USE_LOW_EBDA -a X64 -b RELEASE -t XCODE8 -n 5 | OS: 11.2.3]
...

0:154  0:000  Boot status=0
0:154  0:000  Running on: 'X75VB' with board 'X75VB'                                        --> Laptop Asus
0:154  0:000  === [ GetCPUProperties ] ========================
0:154  0:000  CPU Vendor = 756E6547 Model=306A9
0:155  0:000   The CPU supported SSE4.1
0:155  0:000  BrandString = Intel(R) Core(TM) i5-3340M CPU @ 2.70GHz       --> Ivy Bridge with MBP11,3 (Haswell)

...

 

So thanks

[tluck]

4 hours ago, eng_redaesm said:

@Jief_Machak commit 5417624 built with xcode 12.4. booting fine here

same here. all good

[Slice]

@Jief_Machak

I have Qemu-5 and it works if make symbol link

ln -s qemu_portable libs

Is it interesting for you?

 

qemu-system-x86_64.7z

[Jief_Machak]

11 hours ago, Slice said:

ln -s qemu_portable libs

Not sure to understand where is libs.

I guess you'd like an update to a newer version of qemu in qem_portable ?

If yes, there is a little work that consist of changing the dependent library path. You can see them with otool -L [path to executable]. Brew mostly use full path, which is not portable. TO make it portable I collect every libraries needed and change the libraries path with install_name_tool.

In your case, there is also path to @executable_path/../libs, which why you needed a link call libs. Resolving be creating a link won't be enough to make it portable as there is reference to some libraries in /opt/local/lib.

I'm doing it now.

 

Funny detail : to get the right library version, I used brew. To compile qemu, here is all the dependency installed by brew 

==> Installing dependencies for qemu: libtool, ninja, meson, gettext, libffi, pcre, glib, autoconf, automake, gnu-sed, bdw-gc, libunistring, guile, libidn2, libtasn1, nettle, p11-kit, libevent, c-ares, jemalloc, libev, nghttp2, unbound, gnutls, jpeg, libpng, libssh, libusb, lzo, ncurses, pixman, snappy and vde

knowing that in the end about 20 libraries are enough...

 

I'll commit it when it's done.

[Matgen84]

Hi @Slice

I build latest commit f2dab6c. I don't understand why GitHub web site  don't show the new master for Opencorepkg SubModule in Cloverbootloader directory. Sorry for my bad English.

 

Spoiler

 

When I compile, the update of Opencorepkg appears

 

From https://github.com/CloverHackyColor/CloverBootloader
   541762427..f2dab6c9d  master     -> origin/master
From https://github.com/CloverHackyColor/OpenCorePkg
77af4483..444a41b1  master     -> origin/master

HEAD is now at 444a41b1 resolve CPU names conflict
HEAD is now at f2dab6c9d remove conflict with ProcessorInfo

 

Edited April 17, 2021 by Matgen84

[Slice]

I always check if OpenCorePkg is on master branch and make git pull manually. I also don't know how to do this automatically.

[Jief_Machak]

23 hours ago, Slice said:

I always check if OpenCorePkg is on master branch and make git pull manually. I also don't know how to do this automatically.

Not sure either. Annoying, I agree.

Maybe I'll do a small fetch_and_pull.sh ?

[Matgen84]

3 minutes ago, Jief_Machak said:

Not sure either. Annoying, I agree.

Maybe I'll do a small fetch_and_pull.sh ?

Thanks @Jief_Machak I think it is neccessary for those who git clone a new repo

CloverHackyColor/CloverBootloader

 

Spoiler

 

CloverHackyColor/OpencorePkg

 

Spoiler

 

Strangely yesterday, CloverHackyColor/OpencorePkg is putting on "444a41b" instead of "master" branch. Today, no problem.

[Jief_Machak]

On 4/16/2021 at 9:05 PM, Slice said:

@Jief_Machak

I have Qemu-5 and it works if make symbol link

ln -s qemu_portable libs

Is it interesting for you?

 

qemu-system-x86_64.7z 2.64 MB · 17 downloads

How did you get that version ?

Compiled yourself ? With brew ?

[Slice]

38 minutes ago, Jief_Machak said:

How did you get that version ?

Compiled yourself ? With brew ?

No, I was using MacPort. Qemu's libs are mostly in /opt/local/... but your libraries from qemu_portable are good enough.

[Jief_Machak]

Just now, Slice said:

No, I was using MacPort. Qemu's libs are mostly in /opt/local/... but your libraries from qemu_portable are good enough.

MacPort ? For an unknown reason, I thought brew took over and didn't use MacPorts in years.

I'll try.

I am making a script that can take an executable, copy it to a folder and copy all it's library dependency and adjust each library path. Because libs also have path to other libs, doing it manually can be super-annoying and time consuming.

But currently, my script only works if path to library are absolute. That's why I want to compile the same version as you and update the script.

On 4/17/2021 at 9:14 AM, Jief_Machak said:

==> Installing dependencies for qemu: libtool, ninja, meson, gettext, libffi, pcre, glib, autoconf, automake, gnu-sed, bdw-gc, libunistring, guile, libidn2, libtasn1, nettle, p11-kit, libevent, c-ares, jemalloc, libev, nghttp2, unbound, gnutls, jpeg, libpng, libssh, libusb, lzo, ncurses, pixman, snappy and vde

 

Will see if MacPort has more sensible dependency graph ! With brew, it took 2 hours to compile Qemu, bcause of all these dependencies.

[Slice]

I have long experience with MacPort creating libs for Wine. And now for Qemu I just already have mostly all of them.

[Jief_Machak]

New qemu version committed.

A script "copy_and_make_portable" is committed in the qemu folder. I can be used for anything. I used it for having gcc from brew also portable. And other thing I compiled with brew.

I probably should switch back to MacPorts ! Qemu compiled with MacPorts has only 12 dependent libs. With brew it's 24 ! Total is 15MB instead of 24MB. Not bad 

[Jief_Machak]

Interestingly enough, your qemu executable has a longer list of dependent list, including :

@executable_path/../libs/libintl.8.dylib

@executable_path/../libs/libpixman-1.0.dylib

 

The rest is in /opt/local/lib. The macos one are in /usr/lib or /System/Library but we don't need this ones.

 

When I installed with MacPorts, my executable has less dependent libs and none are in @executable_path/../libs/.

Just curiosity, I wonder what's the difference...

[Slice]

The difference can be in default features at configure the qemu. I don't remember if I chosen them.

[Slice]

Yes, your qemu-5.2.0 is working. So 4.2.0 can be eliminated?

[Jief_Machak]

It's not big. I thought it doesn't hurt to keep it there. But I don't know if someone can need it.

[Jief_Machak]

I'm looking at ParseLoadOptions(). It looks like there is the possibility to have a whole config.plist in efi load options.

I can't really imagine someone typing a Clover config.plist in its load options...

What was it for originally ? Does someone still use that ? Should we still support that ?

 

Same question about GetBootFromOption(). If load options start with a clover sign, load options will interpreted as default volume and default loader. I'm not even sure I can enter a clover sign in the load options.

What is it for ? Should we still support that ?

[Slice]

5 hours ago, Jief_Machak said:

I'm looking at ParseLoadOptions(). It looks like there is the possibility to have a whole config.plist in efi load options.

I can't really imagine someone typing a Clover config.plist in its load options...

What was it for originally ? Does someone still use that ? Should we still support that ?

 

Same question about GetBootFromOption(). If load options start with a clover sign, load options will interpreted as default volume and default loader. I'm not even sure I can enter a clover sign in the load options.

What is it for ? Should we still support that ?

Originally it was UEFI BIOS boot option like boot0003 which contains file path to CloverX64.efi and some arbitrary bytes can be interpreted as a name of config.plist.

The idea was press F12 and select

1. Boot Clover with config1.plist

2. Boot Clover with config2.plist

and so on.

But nobody ever used this.

[Jief_Machak]

1 minute ago, Slice said:

Originally it was UEFI BIOS boot option like boot0003 which contains file patch to CloverX64.efi and some arbitrary bytes can be interpreted as a name of config.plist.

The idea was press F12 and select

1. Boot Clover with config1.plist

2. Boot Clover with config2.plist

and so on.

But nobody ever used this.

I understand selecting another config.plist could be interesting in some rare case (but now we can even duplicate the whole Clover folder if we want multiple configuration).

What about ParseLoadOptions() where it seems that we can put a while config.plist IN load options ? Or do I misread the code ?

[Slice]

Whole config.plist? It should be an error.

[Jief_Machak]

10 minutes ago, Slice said:

Whole config.plist? It should be an error.

 

void
ParseLoadOptions (
                  OUT  XStringW* ConfNamePtr,
                  OUT  TagDict** Dict
                  )
{
  CHAR8 *End;
  CHAR8 *Start;
  UINTN TailSize;
  UINTN i;
  CONST CHAR8 *PlistStrings[]  =
  {
    "<?xml",
    "<!DOCTYPE plist",
    "<plist",
    "<dict>",
    "\0"
  };

  UINTN PlistStringsLen;
  *Dict                  = NULL;

  XStringW& ConfName = *ConfNamePtr;

  Start = (CHAR8*)self.getSelfLoadedImage().LoadOptions;
  End   = (CHAR8*)((CHAR8*)self.getSelfLoadedImage().LoadOptions + self.getSelfLoadedImage().LoadOptionsSize);
  while ((Start < End) && ((*Start == ' ') || (*Start == '\\') || (*Start == '/')))
  {
    ++Start;
  }

  TailSize = End - Start;
  //DBG("TailSize = %d\n", TailSize);

  if ((TailSize) <= 0) {
    return;
  }

  for (i = 0; PlistStrings[i][0] != '\0'; i++) {
    PlistStringsLen = AsciiStrLen(PlistStrings[i]);
    //DBG("PlistStrings[%d] = %s\n", i, PlistStrings[i]);
    if (PlistStringsLen < TailSize) {
      if (AsciiStriNCmp(PlistStrings[i], Start, PlistStringsLen)) {
        DBG(" - found plist string = %s, parse XML in LoadOptions\n", PlistStrings[i]);
        if (ParseXML(Start, Dict, TailSize) != EFI_SUCCESS) {
          *Dict = NULL;
          DBG("  - [!] xml in load options is bad\n");
          return;
        }
        return;
      }
    }
  }

  while ((End > Start) && ((*End == ' ') || (*End == '\\') || (*End == '/'))) {
    --End;
  }

  TailSize = End - Start;
  //  DBG("TailSize2 = %d\n", TailSize);

  if (TailSize > 6) {
    if (AsciiStriNCmp(".plist", End - 6, 6)) {
      End      -= 6;
      TailSize -= 6;
      //      DBG("TailSize3 = %d\n", TailSize);
    }
  } else if (TailSize <= 0) {
    return;
  }

  ConfName.strncpy(Start, TailSize + 1);
}

In this function, there is :

  Start = (CHAR8*)self.getSelfLoadedImage().LoadOptions;

and then :

        if (ParseXML(Start, Dict, TailSize) != EFI_SUCCESS) {

The dict generated is store in gConfigDict[1] by the caller (in main.cpp).

I imagine the purpose is NOT to put the whole thing, of course.

The caller check a tag "Unite". And if "Unite" is false, gConfigDict[0] is not read. So it looks like a way of replacing or uniting config.plist with an xml passed into load options ?

Could I simplify and drop that support ? (and re introduce something if a user is asking, one day).

Edited April 20, 2021 by Jief_Machak