Project

General

Profile

Actions
Copy link

Patch #41094

open

Migrate System Tests from Selenium to Playwright

Added by Mizuki ISHIKAWA 13 days ago. Updated 12 days ago.

Status:
New
Priority:
Normal
Assignee:
-
Category:
Code cleanup/refactoring
Target version:
-
Start date:
Due date:
% Done:

0%

Estimated time:

Description

Currently, Redmine executes system tests using Selenium. I suggestion to switch to Playwright for running these tests.

Playwright can be used as a driver starting from Rails 7.1.
https://github.com/rails/rails/pull/48950

Before running the tests, several commands need to be executed:
  • yarn install
  • npx playwright install
  • npx playwright install-deps
Benefits of using Playwright: Drawbacks of using Playwright:
  • Migration from the existing Selenium environment is required (not needed if using the playwright_or_selenium.patch)
  • Commands to set up Playwright need to be executed before running the tests

I have attached two patches: playwright.patch and playwright_or_selenium.patch. The playwright.patch replaces Selenium with Playwright, while the playwright_or_selenium.patch switches the driver based on the ENV['USE_PLAYWRIGHT']. Please let us know your thoughts on which option is preferable.

Files

Download all files
playwright_or_selenium.patch (72.2 KB) playwright_or_selenium.patch Mizuki ISHIKAWA, 2024-08-06 03:03
playwright.patch (70.9 KB) playwright.patch Mizuki ISHIKAWA, 2024-08-06 03:03
playwright_or_selenium_v2.patch (72.2 KB) playwright_or_selenium_v2.patch Mizuki ISHIKAWA, 2024-08-06 03:06
Actions
Copy link
 #2

Updated by Mizuki ISHIKAWA 13 days ago

tests run with Playwright:

$ USE_PLAYWRIGHT=true rake test TEST=test/system/
To run the system test using Playwright, the following commands are required in advance.
`yarn install`
`npx playwright install`
`npx playwright install-deps`

Run options: --seed 56757

# Running:

Capybara starting Puma...
* Version 6.4.2, codename: The Eagle of Durango
* Min threads: 0, max threads: 4
* Listening on http://127.0.0.1:41937
.......[Screenshot Image]: /var/lib/redmine/tmp/downloads/failures_test_inline_autocomplete_for_issues_with_double_hash_keep_syntax.png
E

Error:
InlineAutocompleteSystemTest#test_inline_autocomplete_for_issues_with_double_hash_keep_syntax:
Capybara::ElementNotFound: Unable to find visible css ".tribute-container" 
    test/system/inline_autocomplete_test.rb:54:in `test_inline_autocomplete_for_issues_with_double_hash_keep_syntax'

bin/rails test test/system/inline_autocomplete_test.rb:47

.......................................[WARNING] Execution context was destroyed, most likely because of a navigation
Call log:

...[WARNING] Execution context was destroyed, most likely because of a navigation
Call log:

............

Finished in 100.760690s, 0.6153 runs/s, 3.7812 assertions/s.
62 runs, 381 assertions, 0 failures, 1 errors, 0 skips

InlineAutocompleteSystemTest#test_inline_autocomplete_for_issues_with_double_hash_keep_syntax test seems to fail in selenium too.

Actions
Copy link
 #3

Updated by Ko Nagase 12 days ago

+1 for playwright_or_selenium option, because some plugins may still need selenium setup in redmine core side.
I confirmed that playwright_or_selenium_v2.patch doesn't affect selenium test results.

In addition of the above error which seems to be same as #40683,
I confirmed the following failure on both playwright and selenium tests,
but I guess that it comes from my test environment (macOS arm64, ruby3.2, PostgreSQL14).

Failure:
IssuesSystemTest#test_create_issue_with_watchers [test/system/issues_test.rb:137]:
--- expected
+++ actual
@@ -1 +1 @@
-["Dave Lopper", "Some Watcher"]
+["Dave Lopper", "John Smith", "Some Watcher"]

Actions
Copy link

Also available in: Atom PDF