Redmine

In Issue #42688, we enabled system tests in GitHub CI (GitHub Actions). At that time, some tests were still unstable and could fail randomly. To avoid making the whole test run fail because of them, we set continue-on-error: true on the system test step.

Five months have passed. Thanks to later improvements, I think system tests are now mostly stable. Therefore, this patch removes continue-on-error: true from the system test job in GitHub CI.

As a result, if a system test fails, the whole workflow will fail.

diff --git a/.github/workflows/tests.yml b/.github/workflows/tests.yml
index 8807e67e0..e980899d6 100644
--- a/.github/workflows/tests.yml
+++ b/.github/workflows/tests.yml
@@ -90,9 +90,6 @@ jobs:
         run: bin/rails test:system
         env:
           GOOGLE_CHROME_OPTS_ARGS: headless,disable-gpu,no-sandbox,disable-dev-shm-usage
-        # System tests might still be a bit unstable, so for now, even if a system test fails,
-        # output the results and consider the overall test as successful.
-        continue-on-error: true

       - name: Upload system test screenshots
         if: always()

In the last three months, the following tests have failed randomly:

https://github.com/redmine/redmine/actions/runs/17997977198/job/51201043679

Failure:
IssuesSystemTest#test_issue_list_with_default_totalable_columns [test/system/issues_test.rb:550]:
Expected false to be truthy.

bin/rails test test/system/issues_test.rb:535

https://github.com/redmine/redmine/actions/runs/17524925619/job/49774056877

Error:
IssuesSystemTest#test_preview_issue_description:
Selenium::WebDriver::Error::UnknownError: unknown error: unhandled inspector error:
(snip)
bin/rails test test/system/issues_test.rb:199

https://github.com/redmine/redmine/actions/runs/16862095340/job/47763775869

Failure:
IssuesSystemTest#test_bulk_edit [test/system/issues_test.rb:427]:
Expected false to be truthy.

rails test test/system/issues_test.rb:403

I would like to create issues and fix these unstable tests when they fail in the future.