Adjust workflow job runner choices for better efficiency #7775
Conversation
mhucka
force-pushed
the
mh-update-gh-runners
branch
from
d4e2951 to
1c5723d
Compare
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #7775 +/- ##
==========================================
- Coverage 99.57% 99.56% -0.01%
==========================================
Files 1102 1102
Lines 98425 98425
==========================================
- Hits 98006 98000 -6
- Misses 419 425 +6 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
mhucka
force-pushed
the
mh-update-gh-runners
branch
4 times, most recently
from
04bcfc5 to
6ae79f5
Compare
I configured customized larger GitHub-hosted runners for the Quantumlib organization for Linux and Windows. These are 8-core runners; we can afford larger runners, but let's see first how much using 8-cores will speed up the workflows in case there are some other bottlenecks that prevent us from gaining any further advantages. Some of the workflows can also make use of `ubuntu-slim` runners, which are faster to start and cheaper to run. Those slim runners have only 1 core, so I looked for jobs that run inherently single-threaded jobs. Finally, GitHub offers a macos-15-large runner with 12 cores. They're about 10x the price of regular runners, so we want to apply them carefully.
That one fails on ubuntum-slim.
The `macos-15-large` is an Intel runner, whereas `macos-15-xlarge` is an Apple M* runner.
`pytest -n auto` ends up using only half the cores on the job runners. (E.g., on the default GitHub 4-core Linux runners, it ends up using 2 workers; on our new 8-core runners, it uses 4; etc.) Let's tell `pytest` to use more than that.
mhucka
force-pushed
the
mh-update-gh-runners
branch
from
84ac265 to
b7865dc
Compare
github-actions
bot
added
size: S
mhucka
force-pushed
the
mh-update-gh-runners
branch
from
b7865dc to
656a710
Compare
pavoljuhas
left a comment
pavoljuhas
left a comment
There was a problem hiding this comment.
Please see inline comments, otherwise LGTM.
| if: github.repository_owner == 'quantumlib' | ||
| name: Type check | ||
| runs-on: ubuntu-22.04 | ||
| runs-on: ubuntu-slim |
| run: dev_tools/conf/pip-install-minimal-for-pytest-changed-files.sh | ||
| - name: Changed files test | ||
| run: check/pytest-changed-files -n auto | ||
| run: check/pytest-changed-files -n $(nproc) |
There was a problem hiding this comment.
The pytest -n option accepts also the "logical" value which uses all cores on my Debian box ("auto" uses half of them). We should use "logical" here and below to avoid OS dependent commands.
| run: check/pytest-changed-files -n $(nproc) | |
| run: check/pytest-changed-files -n logical |
| doc_test: | ||
| if: github.repository_owner == 'quantumlib' | ||
| name: Doc test | ||
| runs-on: ubuntu-22.04 |
There was a problem hiding this comment.
Doc test is quite quick and fails very rarely. I think we can have ubuntu-slim here even if it slows this one down.
| nbformat: | ||
| if: github.repository_owner == 'quantumlib' | ||
| name: Notebook formatting | ||
| runs-on: ubuntu-22.04 |
There was a problem hiding this comment.
This jobs is done under 15s, let's use ubuntu-slim here.
| shellcheck: | ||
| if: github.repository_owner == 'quantumlib' | ||
| name: Shell check | ||
| runs-on: ubuntu-22.04 |
There was a problem hiding this comment.
I have noticed your comment in c2c8d5b.
Did shellcheck fail because it is not installed on ubuntu-slim or is there a newer version that found errors?
| pip-compile: | ||
| if: github.repository_owner == 'quantumlib' | ||
| name: Check consistency of requirements | ||
| runs-on: ubuntu-22.04 |
There was a problem hiding this comment.
This one takes ~ 10s. ubuntu-slim?
| ts-build: | ||
| if: github.repository_owner == 'quantumlib' | ||
| name: Bundle file consistency | ||
| runs-on: ubuntu-22.04 |
There was a problem hiding this comment.
All ts-something jobs here and below are fast and rarely fail, let us run them on ubuntu-slim.
2 participants
After confirming with higher levels of our org that we have the budget for it, I configured customized larger GitHub-hosted runners for the Quantumlib organization. These new runners are 8-core runners for Linux and Windows. (GitHub offers even larger runners, but let's see first how much using 8-cores will speed up the workflows in case there are some other bottlenecks that prevent us from gaining any further advantages.)
By contrast, some of the workflows can also make use of
ubuntu-slimrunners, which are faster to start and cheaper to run. As I understand it, those slim runners have only 1 core, so I looked for jobs that don't do things that can be parallelized. The slim runners take longer to run the jobs, but they're priced at 1/8 the cost per minute of the regular runners, and the particular jobs they're being used for are not bottlenecks; thus, the longer execution of these small jobs doesn't affect the overall workflow time.GitHub offers a macos-15-large runner with 12 cores. They're about 10x the price of regular runners, so we want to apply them carefully.
Finally, I found that the
pytest -n autoflag does not make full use of the available vCPUS. It seems to use only 1/2 of the total available. So to make full use of the runners, I changed-n autoto use the number of processors available.Demonstration of the speed improvements: first, a run from yesterday without the changes:
Now a run (this PR) with the changes: