Testing

[Jief_Machak]

Hey everyone,

 

I'd like to organize a bit more the testing in Clover. The idea is have a "team" of people who agree to make some tests just before a release is about to created. A test is a combination of hardware/macOs version/Clover commit. By macOs version, I imply the latest minor of a major version. I don't think there is a need of keeping track of 10.15.1, 10.15.2 etc. for example.

I've looked into few software, but they are a big too big for what we need. You have to create test plan, test cases, test steps, then plan runs...

Instead, I'm thinking about a shared spreadsheet (Google docs, I guess), where everyone has a tab with a list of all their combination (hardware/macOs version). When we think about a release, I'll send you a message and you test each combination and report it in the spreadsheet.

I think that is simple enough, and efficient for what we need.

Of course, there is no commitment of keeping a hardware or a macOs. If you'd like to join with just your current configuration, that's fine. And you can stop after a while, no question will be asked.

 

To go with that, maybe we can have a forum for beta-testers. There is a some intermediary commits that doesn't really need testing. So I think that if we are getting a little bit organized, it'll be also less work for you.

 

Any idea are welcomed.

 

Anyone up for that ? Send me, in private, an email address you've just created (with your insanely name if possible) so I can start a shared spreadsheet.

 

Cheers.

[Jief_Machak]

The spreadsheet I'm building is here : https://docs.google.com/spreadsheets/d/133g9_8ugmaXQimb_or8ITzf_M5_bEY4Do3cN9tenqXY/edit?usp=sharing

Any suggestions ?

[iCanaro]

in my hack list, at the moment the Clevo is excluded, I'm not running any tests
while the AMD X570 platform is absent

T-RyZo Ryzentosh 3950X MSI Unify Arez Vega56 iMapPro1,1

the N71JQ notebook is also currently excluded from testing

 

PS: i assume i can fix them myself right

Edited October 25, 2020 by iCanaro

[SavageAUS]

Are we testing the latest commit to master? Or do we have a specific version from you to test?


Sent from my iPhone using Tapatalk

[Jief_Machak]

53 minutes ago, iCanaro said:

in my hack list, at the moment the Clevo is excluded

No problem, I just quickly copied things from signature, just to have some content.

You can delete the line. Basically, everyone manages his tab in this spreadsheet. If you add column, some formulas may have to be fixed, but I can do that. I was anyway thinking of adding a column for CPU generation (sandy bridge, Kaby Lake, etc.) just to have a quick overview of what hardware diversity we have.

If the Clevo will come back very soon, you can keep it in the list and put N/A in result. If you deleted, you can put back the line anytime.

 

20 minutes ago, SavageAUS said:

Are we testing the latest commit to master?

The idea is to have a test "campaign" when we think we have a "Release candidate".

For example, I'm currently solving a small thing about partition naming when there is more than one installation in the same container. No need for you to test that now. Let's say the next days I'm solving one or 2 other small things and then, nobody report anything. At this time, I'll ask for everyone to test the latest commit for regression. If there's none, we'll put a tag and have a new "version".

If you find something, we fix and we'll go for another round.

 

That's why I think it'll be less work also for you if we get organized. Sometime maybe you'll just have to test the RC commit and that's it. Knowing there sometimes few dozens of commits between versions, it'll save you time.

There is also commits that we know are "intermediary" and doesn't work. Just a way of passing work between devs. These commits of course shouldn't be tested.

 

So basically, I'll tell you here when we think we have a RC and when it's time for a round of tests.

Edited October 25, 2020 by Jief_Machak

[iCanaro]

1 minute ago, Jief_Machak said:

column for CPU generation (sandy bridge, Kaby Lake, etc.) just to have a quick overview of what hardware diversity we have.

great idea

 

i didn't take anything away but added a line

[Jief_Machak]

Below "From here and below, use at your convience", I just copied your signature. You can reformat it, take way more space if you want, to describe better your configuration. You put as much or as little info you want.

In short : this is your tab. There is just few rules that I'll write in a "Documentation tab". One for example is "Do not change the data validation that impose Pass/Fail/N/A". Well, that's almost it !. Put borders, change font, Change colum title, etc. etc. Make it clear and easy for you. Me ? I'll mainly check dashboard...

Oh non, there an other rule : "Do not remove the total at the bottom of the "Config name" column, it's used in a formula.". Saying that, there's another one  "You have to put something in the "Config name" column, because it's used to calculate the total of tests". Is that too many  ? (kidding).

[Jief_Machak]

11 hours ago, mifjpn said:

Thank you for your efforts. Jief

Looking at the first post, we can see that Jief is really good at handling what we want to contribute.

That's a little bit.

In my case, I'm quite used to terminals on Linux, but I just lacked the actual usage of Git-Hub. It's a really easy-to-use function if you look it up while being taught.

And "How to be helped by me." Is a good example.

Really wonderful!

I definitely want to participate.

 

PM me an email address, so I can share the google doc. You can, of course, create a new email address just for that.

[iCanaro]

@Jief_Machak the email has arrived, I assume it is a test submission, otherwise I did not understand where to download Clover 5126   

[Jief_Machak]

@iCanaro 5126 doesn't yet exist. When we think we are almost ready for it, I'll ask here for you all to test. If everyone passes, we release.

That'll probably happens in few days. Before that, your task is to prepare your tab in the spreadsheet. It's easy, just create a line per combination of hardware/macOS version.

[Jief_Machak]

Hi everyone.

 

we are thinking making d0388ee7a71c8ae9482998417fc1c1ff98d25177 (last commit) the version 5126. So let's start a test campaign ! The first one !!

 

I initially thought making one spreadsheet per version, that's why I had a commit column. I think it's best to specialize a spreadsheet for a specific commit.

That's why I removed the commit column. I've replaced it with a CPU generation. For info like Kaby lake, sandy bridge, etc.

The spreadsheet for this RC is : https://docs.google.com/spreadsheets/d/18q-4giKM0BnD_kTvnwaB-NCDprbbOe_6UMoZ32VoOEU/edit#gid=0

I've reseted everything to "to do" except the N/A.

3. 2. 1. goooooooooo !

EDIT : arrrg. Wait for me to supply the efi file.

 

Edited October 28, 2020 by Jief_Machak

[D-an-W]

I'll send you an email address too and help where I can!

[Jief_Machak]

So, I'm still adjusting, trying the find the best way of working together.

To avoid to have to to set sharing permissions all the time, I shared a folder : https://drive.google.com/drive/folders/1zZ3rojetMe0OSMpwOdyFQeYCr9W9o0By?usp=sharing

I put an the efi and the spreadsheet.

I think this time it's not a bad start : you can 321 go !

[Jief_Machak]

Sorry, it was really a23e29cc6506b826273df621855563c3156533c1.

I'm still adjusting and trying to find the best way. Currently, the idea is one spreadsheet with commit in the name, and the efi that goes with it in google drive.

It's best to test everyone the exact same file, to not have variation if different compilation options (or compiler) are used locally.

I've put the binary download link in the spreadsheet.

[D-an-W]

Is there anything I can fill out regarding hardware or just wait until you finalise the design?

[Jief_Machak]

@D-an-W Well, you already did it ?

Desktop - ASUS PRIME H370-A, i7 8700(iMac19,2), 16GB(8GB*2), MSI RX-570 8G OC,macOS Big Sur / macOS Catalina 10.15.7 / Windows 10
Desktop - ASUS PRIME H370-A, i5 9400-F(iMac19,1), 16GB(8GB*2), MSI RX-570 8G OC,macOS Catalina 10.15.7 / Windows 10 /Ubuntu20.04Lts(Moebuntu)

That yours, right ?

There is no requirement to very precise or anything. We won't go as far as having like a grid of that CPU, with that Graphic test, with that SSD were tested.

The only thing that would be nice is to fill the CPU generation with the code name. i7 8700 is Kaby lake (not sure) ? i5 9400 is more recent, I imagine.

I think that CPU generation is the key thing to be able to boot. The other hardware is matter of config, kext, devices properties etc. So it's less important because here, it's currently boot test, not perfect-hackintosh-with-everything-working test.

[D-an-W]

No that wasn't me!

 

My system is in the Signature below...

[Jief_Machak]

7 minutes ago, D-an-W said:

No that wasn't me!

 

My system is in the Signature below...

Ok. I duplicated tab to create them. I deleted it.

Put more detail of your config if you want, but I don't need to.

 

@everyone Fill the build column with the macos build. I think the format like " 10.15.7 (19H2)" is good, don't you ?

@mifjpn Did I see a fail ? If so, we should suspend the tests, fix the problem. We suspend because fixing the problem might create a new commit...

[iCanaro]

using the attached clover in google drive i don't even get to the GUI
tested on Z370 and Z170
with the one compiled alone

CLOVERX64--5125--a23e29cc6.efi

it works for me pretty much in all hacks

 

CLOVERX64--5125--a23e29cc6.efi.zip

[Jief_Machak]

Just now, iCanaro said:

with the one compiled alone

CLOVERX64--5125--a23e29cc6.efi

"Compiled alone", you mean compiled by you ?

[iCanaro]

2 minutes ago, Jief_Machak said:

"Compiled alone", you mean compiled by you ?

yes

damned translators 

[SavageAUS]

Is it commit a23e29cc6506b826273df621855563c3156533c1 you would like tested?

I can't see a download link on sheets.

 

EDIT: Never mind, fount it.

Edited October 28, 2020 by SavageAUS

[Jief_Machak]

Just now, iCanaro said:

damned translators 

Did you need a translator for this one ?

[iCanaro]

3 minutes ago, SavageAUS said:

Is it commit a23e29cc6506b826273df621855563c3156533c1 you would like tested?

I can't see a download link on sheets.

check your mail

[Jief_Machak]

11 minutes ago, SavageAUS said:

Is it commit a23e29cc6506b826273df621855563c3156533c1 you would like tested?

I can't see a download link on sheets.

On the dashboard. But wait, this efi just crash. I guess a compilation problem. Let me fix that. I'll get back to you all.