Skip to content

Commit

Permalink
ci(win test): deal better with hanging tests
Browse files Browse the repository at this point in the history
When tests hang, we do not see anything in the output, and the job just
times out after 6 hours (!). To add insult to injury, the timeout
prevents the `failed-tests-windows` artifact from being uploaded that is
supposed to help with diagnosing the issue better.

So let's first establish a generous timeout of 20 minutes for the
`win test` matrix jobs (they typically take less than 10 minutes),
and then handle the timed-out tests in addition to the failed ones,
using the presence of a `.out` file in `t/test-results/` combined with
the absence of a `.exit` file as a strong indicator that the
corresponding test timed out.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
  • Loading branch information
dscho committed Dec 19, 2023
1 parent 624eb90 commit 848ed41
Show file tree
Hide file tree
Showing 2 changed files with 20 additions and 3 deletions.
8 changes: 7 additions & 1 deletion .github/workflows/main.yml
Original file line number Diff line number Diff line change
Expand Up @@ -149,13 +149,19 @@ jobs:
run: tar xf artifacts.tar.gz && tar xf tracked.tar.gz
- uses: git-for-windows/setup-git-for-windows-sdk@v1
- name: test
id: test
shell: bash
timeout-minutes: 20
run: . /etc/profile && ci/run-test-slice.sh ${{matrix.nr}} 10
- name: handle timed-out tests
if: failure() && steps.test.outcome == 'failure' && env.FAILED_TEST_ARTIFACTS == ''
shell: bash
run: . /etc/profile && . ci/lib.sh && { handle_failed_tests || test $? = 1; }
- name: print test failures
if: failure() && env.FAILED_TEST_ARTIFACTS != ''
shell: bash
run: ci/print-test-failures.sh
- name: Upload failed tests' directories
- name: Upload failed/timed-out tests' directories
if: failure() && env.FAILED_TEST_ARTIFACTS != ''
uses: actions/upload-artifact@v3
with:
Expand Down
15 changes: 13 additions & 2 deletions ci/lib.sh
Original file line number Diff line number Diff line change
Expand Up @@ -180,15 +180,26 @@ handle_failed_tests () {
}

create_failed_test_artifacts () {
# Handle timed-out tests
for test_out in t/test-results/*.out
do
test -f "${test_out%.out}.exit" ||
echo timed-out >"${test_out%.out}.exit"
done

mkdir -p t/failed-test-artifacts

for test_exit in t/test-results/*.exit
do
test 0 != "$(cat "$test_exit")" || continue
case "$(cat "$test_exit")" in
0) continue;;
timed-out) label="Timed-out";;
*) label="Failed";;
esac

test_name="${test_exit%.exit}"
test_name="${test_name##*/}"
printf "\\e[33m\\e[1m=== Failed test: ${test_name} ===\\e[m\\n"
printf "\\e[33m\\e[1m=== ${label} test: ${test_name} ===\\e[m\\n"
echo "The full logs are in the 'print test failures' step below."
echo "See also the 'failed-tests-*' artifacts attached to this run."
cat "t/test-results/$test_name.markup"
Expand Down

0 comments on commit 848ed41

Please sign in to comment.