Redmine

When a test runner is invoked and the tests include either test/integration/lib/redmine/hook_test.rb or test/unit/lib/redmine/hook_test.rb, there is a side effect that all hooks classes / listeners are cleared. It is probably related to hook listeners being stored in class variables.

This may affect other tests, but most importantly, it makes it difficult to verify redmine core integrity with plugins in place. While plugins' patches are still applied, the listeners are not guaranteed and the results differ between e.g. rake test and rake test:functionals.

More info about the context below.

Although this might not be necessarily a defect - maybe core tests are not supposed to run with plugins - the consequences of plugin developers not running the core tests are terrible and affect the reputation of whole Redmine ecosystem.

• We have run Redmine core tests with plugins we use.
• It's even difficult to find a plugin that doesn't break anything in the core.
• A logical objection would be "plugins modify core functionality, so it's normal that core tests are not passing". But this turned out NOT to be the case. In fact:
• Around 50% of the errors and failures are that the implementation is not robust enough to withstand testing.
• And the other 50% are user-reproducible bugs, which would have been discovered if core tests had been run.
• In the end, we have not observed any error or failure caused by intentional plugin behavior.

Based on your feedback, I can provide specific examples and will be happy to update plugin_tutorial to encourage core testing.