Mint - Run subsequent system tests even if a test fails

[iternity-dotcom]

Is your feature request related to a problem? Please describe.
Whenever the Mint test suite is executed, a single failing test in minio-java or minio-go causes all the remaining tests from the same "test package" to be skipped. The disadvantage is, that you don't see all failing tests but only the first failing test (at least in minio-java and minio-go tests). Ok the advantage is, that you don't need to run the whole test suite to see if "something" is broken.

Describe the solution you'd like
We suggest to introduce an optional runtime variable RUN_ON_FAIL.
By setting the variable RUN_ON_FAIL=1 all tests from all packages should be executed even if a preceeding tests is failing.

Describe alternatives you've considered
Changing the default behavior to ALWAYS run every test independent of failing tests.
This would change the default behavior of the Mint test suite.

Additional context
We implemented this feature for minio-go and minio-java and we will create appropriate PRs and link them to this issue, so that there is a single place for discussions for this.

[iternity-dotcom]

Also refering to a discussion in slack: https://minio.slack.com/archives/C3NDUB8UA/p1615891367010200
@kannappanr @GarimaKapoor

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed after 15 days if no further activity occurs. Thank you for your contributions.

[iternity-dotcom]

There are still some open PRs related to this issue.

-> minio/minio-go#1475
-> minio/minio-java#1180

And a change in minio-java that needs to be reverted: 721525f

We will rebase those PRs and add a new one for minio-java.

[andrebruch]

The two PRs are now up to date:
minio/minio-go#1475
minio/minio-java#1180

The change in the minio-java call (721525f) does not need to be reverted.
I passed the variable RUN_ON_FAIL as environment variable and additionally (that was the error) as parameter to minio-java.