Patch #41094
open
Migrate System Tests from Selenium to Playwright
Added by Mizuki ISHIKAWA 13 days ago.
Updated 12 days ago.
Category:
Code cleanup/refactoring
Description
Currently, Redmine executes system tests using Selenium. I suggestion to switch to Playwright for running these tests.
Playwright can be used as a driver starting from Rails 7.1.
https://github.com/rails/rails/pull/48950
Before running the tests, several commands need to be executed:
yarn install
npx playwright install
npx playwright install-deps
Benefits of using Playwright:
Drawbacks of using Playwright:
- Migration from the existing Selenium environment is required (not needed if using the playwright_or_selenium.patch)
- Commands to set up Playwright need to be executed before running the tests
I have attached two patches: playwright.patch and playwright_or_selenium.patch. The playwright.patch replaces Selenium with Playwright, while the playwright_or_selenium.patch switches the driver based on the ENV['USE_PLAYWRIGHT']. Please let us know your thoughts on which option is preferable.
Files
tests run with Playwright:
$ USE_PLAYWRIGHT=true rake test TEST=test/system/
To run the system test using Playwright, the following commands are required in advance.
`yarn install`
`npx playwright install`
`npx playwright install-deps`
Run options: --seed 56757
# Running:
Capybara starting Puma...
* Version 6.4.2, codename: The Eagle of Durango
* Min threads: 0, max threads: 4
* Listening on http://127.0.0.1:41937
.......[Screenshot Image]: /var/lib/redmine/tmp/downloads/failures_test_inline_autocomplete_for_issues_with_double_hash_keep_syntax.png
E
Error:
InlineAutocompleteSystemTest#test_inline_autocomplete_for_issues_with_double_hash_keep_syntax:
Capybara::ElementNotFound: Unable to find visible css ".tribute-container"
test/system/inline_autocomplete_test.rb:54:in `test_inline_autocomplete_for_issues_with_double_hash_keep_syntax'
bin/rails test test/system/inline_autocomplete_test.rb:47
.......................................[WARNING] Execution context was destroyed, most likely because of a navigation
Call log:
...[WARNING] Execution context was destroyed, most likely because of a navigation
Call log:
............
Finished in 100.760690s, 0.6153 runs/s, 3.7812 assertions/s.
62 runs, 381 assertions, 0 failures, 1 errors, 0 skips
InlineAutocompleteSystemTest#test_inline_autocomplete_for_issues_with_double_hash_keep_syntax test seems to fail in selenium too.
+1 for playwright_or_selenium
option, because some plugins may still need selenium setup in redmine core side.
I confirmed that playwright_or_selenium_v2.patch
doesn't affect selenium test results.
In addition of the above error which seems to be same as #40683,
I confirmed the following failure on both playwright and selenium tests,
but I guess that it comes from my test environment (macOS arm64, ruby3.2, PostgreSQL14).
Failure:
IssuesSystemTest#test_create_issue_with_watchers [test/system/issues_test.rb:137]:
--- expected
+++ actual
@@ -1 +1 @@
-["Dave Lopper", "Some Watcher"]
+["Dave Lopper", "John Smith", "Some Watcher"]
Also available in: Atom
PDF