Bootloader Build_Clover.command, another Script to build standard Clover (or customized)

Slice · August 24, 2016

https://sourceforge.net/p/cloverefiboot/code/3719/

Edit: tried to set the NASM path to ENV(NASM_PREFIX)/nasm in tools_def.txt manually, but it still looks for it in /opt/local/bin/nasm. WTF!!!

Edit2: the problem is not EDK-related, reverted it to the recommended revision 22435 and got the same errors. I smell hardcoded path somewhere...

I meant ENV(NASM_PREFIX)/nasm instead of /opt/local/bin/nasm

Philip Petev · August 24, 2016

I meant ENV(NASM_PREFIX)/nasm instead of /opt/local/bin/nasm

Yeah, yeah, figured that out already. My problem was I had to run ebuild.sh clean after those changes (commenting out the line with /opt/local/bin/nasm and then uncommenting the above line). Now clover can be build correctly, but you two (you and Micky1979) really should decide where the nasm should be installed and change Clover or the script accordingly. Right now, they are sort of incompatible.

Slice · August 24, 2016

Help me, please, in writing such bash script

if [file exists "/opt/local/bin/nasm"] then

NASM_PREFIX="/opt/local/bin"

else if [file exists "(HOME)/src//opt/local/bin"] then

NASM_PREFIX="(HOME)/src//opt/local/bin"

else

NASM_PREFIX=""

fi

And then check for version

$ nasm -v

NASM version 2.11.06 compiled on Dec 4 2014

echo "bad nasm version, please install new one"

cecekpawon · August 24, 2016

Try this l0rd

if [[ -f "/opt/local/bin/nasm" ]]; then
  NASM_PREFIX="/opt/local/bin/nasm"
elif [[ -f "(HOME)/src//opt/local/bin" ]]; then
  NASM_PREFIX="(HOME)/src//opt/local/bin"
else
  NASM_PREFIX=""
fi

#echo $NASM_PREFIX

#NASM_VER=`nasm -v | awk '/version/ {print $3}'`
NASM_VER=`nasm -v | sed -nE 's/^.*version.([0-9\.]+).*$/\1/p'`

#echo $NASM_VER

Micky1979 · August 24, 2016

take a look on my latest beta script some posts ago... it looks for each nasm version already in $PATH and perform a check and decide if good or not. Maybe something can be reused..

EDIT:

restoreIFS() {

IFS=$' \t\n';

}

needNASM() {

restoreIFS

local nasmPath=""

local nasmArray=( $(which -a nasm) )

local needInstall=1

local good=""

if [ ${#nasmArray[@]} -ge "1" ]; then

for i in "${nasmArray[@]}"

do

echo "found nasm v$(${i} -v | grep 'NASM version' | awk '{print $3}') at $(dirname ${i})"

done

# we have a good nasm?

for i in "${nasmArray[@]}"

do

if isNASMGood "${i}"; then

good="${i}"

break

fi

done

if [[ -x "${good}" ]] ; then

# only nasm at index 0 is used!

if [[ "${good}" == "${nasmArray[0]}" ]]; then

echo "nasm is ok.."

else

echo "this one is good:"

echo "${good}"

echo "..but will not be used.."

cp -R "${good}" "${NASM_PREFIX}"/

echo "${good} copied to ${NASM_PREFIX}/!"

fi

else

# no nasm versions suitable for Clover

echo "nasm found, but is not good to build Clover.."

needInstall=0

fi

else

needInstall=0

echo "nasm not found.."

fi

return $needInstall

}

isNASMGood() {

restoreIFS

IFS='.';

local array=($( "${1}" -v | grep 'NASM version' | awk '{print $3}') )

case "${#array[@]}" in

"2" | "3")

if [ "${array[0]}" -ge "3" ]; then

return 0;

fi

if [ "${array[0]}" -eq "2" ] && [ "${array[1]}" -ge "12" ]; then

return 0;

fi

;;

*)

echo "Unknown nasm version format.."

;;

esac

return 1

}

Micky1979 · August 24, 2016

Hi Slice what is the result for you of:

echo $USER

on your build machine??

Micky1979 · August 24, 2016

Slice, take a look on this.

Usage:
(-nb = don't rebuild boot files)

./ebuild.sh -x64 -nb -t XCODE5
./ebuild.sh -ia32 -nb -t XCODE5

result:

build started at:
Thu Aug 25 00:43:00 CEST 2016
finished at
Thu Aug 25 00:43:27 CEST 2016

27 seconds for both architectures (12 for x64 + 11 for ia32. 4 seconds copying every new efi files to the Clover package directories ). Check if is what you need.

ebuild.sh.zip

Philip Petev · August 25, 2016

@Slice: why don't you just change all paths to NASM in Patches_for_EDK2/Conf/tools_def.txt to ENV(NASM_PREFIX)/nasm, so every build script (incl. ebuild.sh) can define for itself which nasm instance to use via that variable?

Slice · August 25, 2016

Hi Slice what is the result for you of:
echo $USER
on your build machine??

Sergeys-iMac:Clover slice$ echo $USER
slice
Sergeys-iMac:Clover slice$

@Slice: why don't you just change all paths to NASM in Patches_for_EDK2/Conf/tools_def.txt to ENV(NASM_PREFIX)/nasm, so every build script (incl. ebuild.sh) can define for itself which nasm instance to use via that variable?

Yes, I assumed this after accepting new ebuild.sh.

NASM_PREFIX is missing forward slash

e.g

NASM_PREFIX=~/src/opt/local/bin/

in tools_def.txt ( unmodified except for make GCC make path )

*_*_*_NASM_PATH = ENV(NASM_PREFIX)nasm

No, slash should be out of prefix

*_*_*_NASM_PATH = ENV(NASM_PREFIX)/nasm

take a look on my latest beta script some posts ago... it looks for each nasm version already in $PATH and perform a check and decide if good or not. Maybe something can be reused..

EDIT:
restoreIFS() {

IFS=$' \t\n';

}

needNASM() {

restoreIFS

local nasmPath=""

local nasmArray=( $(which -a nasm) )

local needInstall=1

local good=""

if [ ${#nasmArray[@]} -ge "1" ]; then

for i in "${nasmArray[@]}"

do

echo "found nasm v$(${i} -v | grep 'NASM version' | awk '{print $3}') at $(dirname ${i})"

done

# we have a good nasm?

for i in "${nasmArray[@]}"

do

if isNASMGood "${i}"; then

good="${i}"

break

fi

done

if [[ -x "${good}" ]] ; then

# only nasm at index 0 is used!

if [[ "${good}" == "${nasmArray[0]}" ]]; then

echo "nasm is ok.."

else

echo "this one is good:"

echo "${good}"

echo "..but will not be used.."

cp -R "${good}" "${NASM_PREFIX}"/

echo "${good} copied to ${NASM_PREFIX}/!"

fi

else

# no nasm versions suitable for Clover

echo "nasm found, but is not good to build Clover.."

needInstall=0

fi

else

needInstall=0

echo "nasm not found.."

fi

return $needInstall

}

isNASMGood() {

restoreIFS

IFS='.';

local array=($( "${1}" -v | grep 'NASM version' | awk '{print $3}') )

case "${#array[@]}" in

"2" | "3")

if [ "${array[0]}" -ge "3" ]; then

return 0;

fi

if [ "${array[0]}" -eq "2" ] && [ "${array[1]}" -ge "12" ]; then

return 0;

fi

;;

*)

echo "Unknown nasm version format.."

;;

esac

return 1

}

Where to apply this?

Philip Petev · August 25, 2016

take a look on my latest beta script some posts ago... it looks for each nasm version already in $PATH and perform a check and decide if good or not. Maybe something can be reused..

EDIT:

restoreIFS() {

IFS=$' \t\n';

}

needNASM() {

restoreIFS

local nasmPath=""

local nasmArray=( $(which -a nasm) )

local needInstall=1

local good=""

if [ ${#nasmArray[@]} -ge "1" ]; then

for i in "${nasmArray[@]}"

do

echo "found nasm v$(${i} -v | grep 'NASM version' | awk '{print $3}') at $(dirname ${i})"

done

# we have a good nasm?

for i in "${nasmArray[@]}"

do

if isNASMGood "${i}"; then

good="${i}"

break

fi

done

if [[ -x "${good}" ]] ; then

# only nasm at index 0 is used!

if [[ "${good}" == "${nasmArray[0]}" ]]; then

echo "nasm is ok.."

else

echo "this one is good:"

echo "${good}"

echo "..but will not be used.."

cp -R "${good}" "${NASM_PREFIX}"/

echo "${good} copied to ${NASM_PREFIX}/!"

fi

else

# no nasm versions suitable for Clover

echo "nasm found, but is not good to build Clover.."

needInstall=0

fi

else

needInstall=0

echo "nasm not found.."

fi

return $needInstall

}

isNASMGood() {

restoreIFS

IFS='.';

local array=($( "${1}" -v | grep 'NASM version' | awk '{print $3}') )

case "${#array[@]}" in

"2" | "3")

if [ "${array[0]}" -ge "3" ]; then

return 0;

fi

if [ "${array[0]}" -eq "2" ] && [ "${array[1]}" -ge "12" ]; then

return 0;

fi

;;

*)

echo "Unknown nasm version format.."

;;

esac

return 1

}

All that would be unnecessary, if the Clover patches are changed according to my proposal (post 112). You have already that variable defined in your current script version, all it takes is to use it.

Micky1979 · August 25, 2016

In buildnasn.sh since Clover already have this script. Need adjustment because was for my script and someting like copy operation is not needed. You can use that this way:

if ! needNASM; then

exit 0

fi

. . otherwise install nsam

philip, we need to ensure is the correct version....

Slice · August 25, 2016

Sergeys-iMac:Clover slice$ ./ebuild.sh -nb
TOOLCHAIN_DIR: /Users/slice/src/edk2/Clover/../../opt/local
found nasm v2.12.01 at /Users/slice/src/edk2/Clover/../../opt/local/bin
found nasm v2.12.02 at /opt/local/bin
found nasm v2.12.01 at /usr/gcc-4.8/bin
found nasm v2.12.02 at /opt/local/bin
found nasm v0.98.40 at /usr/bin
nasm is ok..
Sergeys-iMac:Clover slice$

and finish?!

Philip Petev · August 25, 2016

In buildnasn.sh since Clover already have this script. Need adjustment because was for my script and someting like copy operation is not needed. You can use that this way:

if ! needNASM; then

exit 0

fi

. . otherwise install nsam

philip, we need to ensure is the correct version....

Well, it will be the correct version, doesn't your script always download and install the correct version (if not found any other)?

Slice · August 25, 2016

This script after modification

ebuild.sh.zip

Philip Petev · August 25, 2016

@Slice, the script looks fine, but has one potential problem: the NASM_PREFIX variable should be definitely defined (non-empty) before the execution of ebuild.sh (it doesn't define it), otherwise that part may fail.

Slice · August 25, 2016

@Slice, the script looks fine, but has one potential problem: the NASM_PREFIX variable should be definitely defined (non-empty) before the execution of ebuild.sh (it doesn't define it), otherwise that part may fail.

And it happen.

Guys, I know nothing about scripts, I don't know how to do this correctly.

cecekpawon · August 25, 2016

Yeah.... symlink will do too, but that's only a workaround.

Edit: tried to set the NASM path to ENV(NASM_PREFIX)/nasm in tools_def.txt manually, but it still looks for it in /opt/local/bin/nasm. WTF!!!

Edit2: the problem is not EDK-related, reverted it to the recommended revision 22435 and got the same errors. I smell hardcoded path somewhere...

If you need custom, always edit/compare both "/edk2/Clover/Patches_for_EDK2/Conf/tools_def.txt" and "/edk2/Conf/tools_def.txt". Sometimes that tools_def got tampered by other wrapper / maybe you. Or use this params to specify other conf path. Or do export '$CONF_PATH' & '$NASM_PREFIX' vars for build.

Slice · August 25, 2016

Slice, take a look on this.

Usage:

(-nb = don't rebuild boot files)
./ebuild.sh -x64 -nb -t XCODE5
./ebuild.sh -ia32 -nb -t XCODE5
result:
build started at:
Thu Aug 25 00:43:00 CEST 2016
finished at
Thu Aug 25 00:43:27 CEST 2016
27 seconds for both architectures (12 for x64 + 11 for ia32. 4 seconds copying every new efi files to the Clover package directories ). Check if is what you need.

Anyway the boot file is created and compressing

- Done -
Build end time: 12:53:25, Aug.25 2016
Build total time: 00:02:17

Compressing DUETEFIMainFv.FV ...
Encoding
Compressing DxeCore.efi ...
Encoding
Compressing DxeIpl.efi ...
Encoding
Generate Loader Image ...

Micky1979 · August 25, 2016

Anyway the boot file is created and compressing

- Done -
Build end time: 12:53:25, Aug.25 2016
Build total time: 00:02:17

Compressing DUETEFIMainFv.FV ...
Encoding
Compressing DxeCore.efi ...
Encoding
Compressing DxeIpl.efi ...
Encoding
Generate Loader Image ...

We can do like that also at the beginning of the function:

    if (( $NOBOOTFILES == 0 )); then
      echo Compressing DUETEFIMainFv.FV ...
      "$BASETOOLS_DIR"/LzmaCompress -e -o "${BUILD_DIR}/FV/DUETEFIMAINFV${TARGETARCH}.z" "${BUILD_DIR}/FV/DUETEFIMAINFV${TARGETARCH}.Fv"

      echo Compressing DxeCore.efi ...
      "$BASETOOLS_DIR"/LzmaCompress -e -o "${BUILD_DIR}/FV/DxeMain${TARGETARCH}.z" "$BUILD_DIR_ARCH/DxeCore.efi"

      echo Compressing DxeIpl.efi ...
      "$BASETOOLS_DIR"/LzmaCompress -e -o "${BUILD_DIR}/FV/DxeIpl${TARGETARCH}.z" "$BUILD_DIR_ARCH/DxeIpl.efi"

      echo "Generate Loader Image ..."
    fi

and to the end of the "MainPostBuildScript" when it calls:

DESTDIR="$CLOVER_PKG_DIR"/BootSectors make -C $BOOTHFS

make should find all as before and rebuild only changed stuff. Otherwise the MakeFile is buggie

Sergeys-iMac:Clover slice$ ./ebuild.sh -nb
TOOLCHAIN_DIR: /Users/slice/src/edk2/Clover/../../opt/local
found nasm v2.12.01 at /Users/slice/src/edk2/Clover/../../opt/local/bin
found nasm v2.12.02 at /opt/local/bin
found nasm v2.12.01 at /usr/gcc-4.8/bin
found nasm v2.12.02 at /opt/local/bin
found nasm v0.98.40 at /usr/bin
nasm is ok..
Sergeys-iMac:Clover slice$

and finish?!

Actually my statement check only array[0] and array[1] of 2.12.02 so it says that is ok, but in reality you have a bad nasm at:

found nasm v2.12.01 at /Users/slice/src/edk2/Clover/../../opt/local/bin

that is the first, and this will not be used only if tools-def decide otherwise. "/Users/slice/src/opt/local/bin" is the good place to avoid any problems since added to $PATH by ebuild.sh before any other path. Also calling buildnasm.sh users cannot claim an unwanted installation since "src/opt/local/bin" is part of Clover..... is the best place IMHO, just looks here "et voilà" because already added at beginning of $PATH, no chances to fail and will be a easy rule for all. Also all the scripts in Clover source already seem to stay with that.

Sorry to say that, but now I'm sure you used nasm v2.12.01 until commit of r3718... before change the nasm path in tools-def.txt :blush:

EDIT I'm at the job and cannot do anything other than speak until 6:00 PM

Philip Petev · August 25, 2016

This becomes too complicated. I think it would be better nasm to be present only in TOOLCHAIN_DIR and exactly that version to be used and if that version is not proper, the proper version to be downloaded and installed in TOOLCHAIN_DIR.

Slice · August 25, 2016

I just don't want to double tools in all users homes when I have good one in common folder /opt/local/bin.

There is one problem that the /opt folder has other permission and can't be modified from the script.

NASM_PREFIX is missing forward slash

e.g

NASM_PREFIX=~/src/opt/local/bin/

in tools_def.txt ( unmodified except for make GCC make path )

*_*_*_NASM_PATH = ENV(NASM_PREFIX)nasm

Sorry, you are right. Prefix must contain / else empty prefix causes error like "/nasm"

Good way


Sergeys-iMac:Clover slice$ ./ebuild.sh --module=rEFIt_UEFI/refit.inf
TOOLCHAIN_DIR: /Users/slice/src/edk2/Clover/../../opt/local
NASM_PREFIX: /opt/local/bin/
NASM_VER: 2.12.02
Initializing workspace

...
Building ... /Users/slice/src/edk2/Clover/Library/OpensslLib/OpensslLibNull.inf [X64]
Building ... /Users/slice/src/edk2/MdeModulePkg/Library/DxeNetLib/DxeNetLib.inf [X64]
[CC] main
[CC] menu
[CC] DataHubCpu
[SLINK] CLOVERX64
[DLINK1] CLOVERX64
[MTOC] CLOVERX64
[GENFW] CLOVERX64

- Done -
Build end time: 15:03:10, Aug.25 2016
Build total time: 00:00:23

Sergeys-iMac:Clover slice$

Micky1979 · August 25, 2016

Hi all,

Slice ebuild.sh is almost ok but (HOME) does nothing (should be ${HOME} or $HOME), this is ok here (also corrected the check for nasm):

ebuild.sh.zip

This is the new script v3.2 rc1, and now revert edk2 to the r22435 (EDK2_REV="22435"), auto rebuild BaseTools if different after the update.

Nasm is 2.12.02 (downloaded but not compiled), and can be in /opt/local/bin or inside ~/src/opt/local/bin

A check if repos are online befor sync:

FAST_UPDATE="YES" # or FAST_UPDATE="NO" to be faster

disabled by default.

If the user is "slice", no downloads of any kind, no updates of any kind, no nasm/gettext/gcc (or other) are installed or touched...it simply cd to Clover folder and the menù for Slice is simple:

Please enter your choice: 
1) check revision
2) build with ./ebuild.sh -nb
3) build with ./ebuild.sh --module=rEFIt_UEFI/refit.inf
4) build all (boot3, 6 and 7)
5) build all with FORCEREBUILD (boot3, 6 and 7)
6) build pkg
7) build iso
8) build pkg+iso
#?

also never exit (always call this menù after each operations.. to be fast). if you like and need adjustment.. tell me.

relative scripting:

            "build with ./ebuild.sh -nb")
                printHeader 'ebuild.sh -nb'
                cd "${DIR_MAIN}"/edk2/Clover
                ./ebuild.sh -nb
                break
            ;;
            "build with ./ebuild.sh --module=rEFIt_UEFI/refit.inf")
                cd "${DIR_MAIN}"/edk2/Clover
                printHeader 'ebuild.sh --module=rEFIt_UEFI/refit.inf'
                ./ebuild.sh --module=rEFIt_UEFI/refit.inf
                break
            ;;
            "build all (boot3, 6 and 7)")
                cd "${DIR_MAIN}"/edk2/Clover
                printHeader 'boot6'
                ./ebuild.sh -x64 -D NO_GRUB_DRIVERS_EMBEDDED -D CHECK_FLAGS -t XCODE5
                printHeader 'boot7'
                ./ebuild.sh -mc --no-usb -D NO_GRUB_DRIVERS_EMBEDDED -D CHECK_FLAGS -t XCODE5
                printHeader 'boot3'
                ./ebuild.sh -ia32 -D NO_GRUB_DRIVERS_EMBEDDED -D CHECK_FLAGS -t XCODE5
                break
            ;;
            "build all with FORCEREBUILD (boot3, 6 and 7)")
                cd "${DIR_MAIN}"/edk2/Clover
                printHeader 'boot6'
                ./ebuild.sh -fr -x64 -D NO_GRUB_DRIVERS_EMBEDDED -D CHECK_FLAGS -t XCODE5
                printHeader 'boot7'
                ./ebuild.sh -fr -mc --no-usb -D NO_GRUB_DRIVERS_EMBEDDED -D CHECK_FLAGS -t XCODE5
                printHeader 'boot3'
                ./ebuild.sh -fr -ia32 -D NO_GRUB_DRIVERS_EMBEDDED -D CHECK_FLAGS -t XCODE5
                break
            ;;
            "build pkg")
                cd "${DIR_MAIN}"/edk2/Clover
                printHeader 'make pkg'
                make pkg
                break
            ;;
            "build iso")
                cd "${DIR_MAIN}"/edk2/Clover
                printHeader 'make iso'
                make iso
                break
            ;;
            "build pkg+iso")
                cd "${DIR_MAIN}"/edk2/Clover
                printHeader 'make pkg + make iso'
                make pkg
                make iso
                break
            ;;

For all others it looks like before:

Please enter your choice: 
1) check revision
2) update Clover only (no building)
3) update & build Clover
4) run my script on the source
5) build existing revision (no update, standard build)
6) build existing revision with FORCEREBUILD (no update, standard build)
7) build existing revision with custom macros enabled
8) build Clover from scratch (delete /Users/Micky1979/src)
9) info and limitations about this script
#?

if you change nasm path, please run option 6 once, i.e build existing revision with FORCEREBUILD (no update, standard build).

The new script to be working need the above ebuild.sh, so replace it after the update. Hope Slice will accept it.

Edited August 25, 2016 by Micky1979
attachment removed, look at the Download section

Slice · August 25, 2016

No doubt! I accept it in rev 3725. Great work!

Micky1979 · August 25, 2016

Micky1979 · August 25, 2016

New script available here: http://www.insanelymac.com/forum/files/file/589-build-clovercommand/

(also uncommented "edk2" function that was disabled by mistake here due to personal tests made before upload there. now is ok)

EDIT

updated also the following

By Micky1979 based on Slice, Zenith432, STLVNUB, JrCs, cecekpawon, cvad,
Rehabman, philip_petev

# Big thanks to the following testers:
# droples, Riley Freeman, pico joe, fantomas1, Fljagd, calibre, Mork vom Ork
# and all others (I'll be happy to increase this list)

Bootloader Build_Clover.command, another Script to build standard Clover (or customized)

Recommended Posts

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites