Redmine

• Only for things I watch or I am assigned to;
• Only for things I watch or I am owner of.

The application log not show any error.

Do you use any plugin? Please see Submissions.

Marius BALTEANU wrote:

Do you use any plugin?

yes, but I have also tried with a fresh installation without plugin. I'm on Bitnami redmine stack.
If I set For any event on all my projects in my setting, all works as expected.

Thanks in advance.

Marius BALTEANU wrote:

Please see Submissions.

Redmine version 4.0.5.stable
Ruby version 2.5.7-p206 (2019-10-01) [x64-mingw32]
Rails version 5.2.3
Environment production
Database adapter Mysql2
Mailer queue ActiveJob::QueueAdapters::AsyncAdapter
Mailer delivery smtp

pasquale [:dedalus] wrote:

Marius BALTEANU wrote:

Do you use any plugin?

yes, but I have also tried with a fresh installation without plugin. I'm on Bitnami redmine stack.
If I set For any event on all my projects in my setting, all works as expected.

Thanks in advance.

Can you try with a non Bitnami Redmine stack? There are no such problems in the 4.0.* releases and the functionality is covered by tests (ex: https://github.com/redmine/redmine/blob/master/test/unit/mailer_test.rb#L399)

Marius BALTEANU wrote:

Can you try with a non Bitnami Redmine stack? There are no such problems in the 4.0.* releases and the functionality is covered by tests (ex: https://github.com/redmine/redmine/blob/master/test/unit/mailer_test.rb#L399)

I try asap and report the results here.

Hi Marius, we have make further investigation on my bitnami redmine stack installation. The behavior is very strange: it seem that the issue is related to the number of members of the project.

If I'm a watcher (not assignee or author of the ticket), I receive email notification if the member of the project are <=30.
I have observed that in this magic number limit, are computed also the groups if I set the member of the project via the related groups. For esample if I have as members of the projects 24 users in 5 groups (24+5=29), I not receive the email notification. If I have 30 users and 0 groups, I receive the email notification.

I cannot find a related line of code to this magic number in redmine github repository...

Before start testing, we have projects usually with 6 groups and 40\50 members (also locked ones members).
Reading the log I see that the mailer detects that I'm watcher (my user id =3), but not send me the email after that.

Started POST "/redmine/projects/281f3ca1-9edb-4443-a780-f95e0a5375eo/issues" for EDITED at 2019-10-25 08:54:19 +0200
Processing by IssuesController#create as HTML
  Parameters: {"utf8"=>"✓", "authenticity_token"=>"EDITED", "form_update_triggered_by"=>"", "issue"=>{"is_private"=>"0", "tracker_id"=>"1", "subject"=>"test mail", "description"=>"test mail", "status_id"=>"10", "priority_id"=>"3", "assigned_to_id"=>"1", "category_id"=>"", "fixed_version_id"=>"", "parent_issue_id"=>"", "start_date"=>"2019-10-25", "due_date"=>"", "estimated_hours"=>"", "done_ratio"=>"0", "custom_field_values"=>{"32"=>[""], "30"=>"0", "31"=>"0", "13"=>"0", "24"=>"", "25"=>"0", "10"=>"100", "9"=>[""], "6"=>"", "8"=>"", "3"=>"", "27"=>"", "26"=>"", "21"=>""}, "watcher_user_ids"=>["", "3"]}, "was_default_status"=>"10", "commit"=>"Crea", "project_id"=>"281f3ca1-9edb-4443-a780-f95e0a5375eo"}
  Current user: fabio (id=1)
[ActiveJob] Enqueued ActionMailer::DeliveryJob (Job ID: 46277ca8-57a0-4908-a3e6-bf7096e13211) to Async(mailers) with arguments: "Mailer", "issue_add", "deliver_now", #<GlobalID:0x0000000012d8e570 @uri=#<URI::GID gid://redmine-app/User/1>>, #<GlobalID:0x0000000012d8da80 @uri=#<URI::GID gid://redmine-app/Issue/15110>>
[ActiveJob] Enqueued ActionMailer::DeliveryJob (Job ID: 31e43fc3-d1db-44d1-8ec2-732ded7b209a) to Async(mailers) with arguments: "Mailer", "issue_add", "deliver_now", #<GlobalID:0x0000000012d67538 @uri=#<URI::GID gid://redmine-app/User/4>>, #<GlobalID:0x0000000012d66278 @uri=#<URI::GID gid://redmine-app/Issue/15110>>
[ActiveJob] Enqueued ActionMailer::DeliveryJob (Job ID: a804b4b4-a0bf-4252-9c2c-8d13a975edc2) to Async(mailers) with arguments: "Mailer", "issue_add", "deliver_now", #<GlobalID:0x0000000012d43340 @uri=#<URI::GID gid://redmine-app/User/7>>, #<GlobalID:0x0000000012d42300 @uri=#<URI::GID gid://redmine-app/Issue/15110>>
[ActiveJob] Enqueued ActionMailer::DeliveryJob (Job ID: 583bace0-30e7-4b79-8f89-72263fc81527) to Async(mailers) with arguments: "Mailer", "issue_add", "deliver_now", #<GlobalID:0x0000000012d28040 @uri=#<URI::GID gid://redmine-app/User/16>>, #<GlobalID:0x0000000012d03830 @uri=#<URI::GID gid://redmine-app/Issue/15110>>
[ActiveJob] Enqueued ActionMailer::DeliveryJob (Job ID: 2135b1ea-73c3-42bb-9e2f-d3f6765bb59c) to Async(mailers) with arguments: "Mailer", "issue_add", "deliver_now", #<GlobalID:0x0000000012ce9750 @uri=#<URI::GID gid://redmine-app/User/20>>, #<GlobalID:0x0000000012ce8b98 @uri=#<URI::GID gid://redmine-app/Issue/15110>>
[ActiveJob] Enqueued ActionMailer::DeliveryJob (Job ID: aea90641-92d3-47ef-9239-9b9cc415b3be) to Async(mailers) with arguments: "Mailer", "issue_add", "deliver_now", #<GlobalID:0x0000000012cca530 @uri=#<URI::GID gid://redmine-app/User/22>>, #<GlobalID:0x0000000012cc9630 @uri=#<URI::GID gid://redmine-app/Issue/15110>>
[ActiveJob] Enqueued ActionMailer::DeliveryJob (Job ID: 886d174f-1276-46e2-90ac-d58f93e2a23d) to Async(mailers) with arguments: "Mailer", "issue_add", "deliver_now", #<GlobalID:0x0000000012c9aab0 @uri=#<URI::GID gid://redmine-app/User/23>>, #<GlobalID:0x0000000012c99a20 @uri=#<URI::GID gid://redmine-app/Issue/15110>>
[ActiveJob] Enqueued ActionMailer::DeliveryJob (Job ID: e35a61de-d94f-4aa6-a368-5e6a435e683e) to Async(mailers) with arguments: "Mailer", "issue_add", "deliver_now", #<GlobalID:0x0000000012c89d50 @uri=#<URI::GID gid://redmine-app/User/26>>, #<GlobalID:0x0000000012c89418 @uri=#<URI::GID gid://redmine-app/Issue/15110>>
[ActiveJob] Enqueued ActionMailer::DeliveryJob (Job ID: 57333403-3cfe-4968-8297-3e1db1b03dc8) to Async(mailers) with arguments: "Mailer", "issue_add", "deliver_now", #<GlobalID:0x0000000012c5ee20 @uri=#<URI::GID gid://redmine-app/User/39>>, #<GlobalID:0x0000000012c5dfe8 @uri=#<URI::GID gid://redmine-app/Issue/15110>>
[ActiveJob] Enqueued ActionMailer::DeliveryJob (Job ID: 1274a7be-893e-42a9-b2db-29e68efa638e) to Async(mailers) with arguments: "Mailer", "issue_add", "deliver_now", #<GlobalID:0x0000000012c479c8 @uri=#<URI::GID gid://redmine-app/User/40>>, #<GlobalID:0x0000000012c47068 @uri=#<URI::GID gid://redmine-app/Issue/15110>>
[ActiveJob] Enqueued ActionMailer::DeliveryJob (Job ID: 3032de8f-488f-455e-98ce-2fde41b95074) to Async(mailers) with arguments: "Mailer", "issue_add", "deliver_now", #<GlobalID:0x0000000012c05b68 @uri=#<URI::GID gid://redmine-app/User/57>>, #<GlobalID:0x0000000012c05168 @uri=#<URI::GID gid://redmine-app/Issue/15110>>
[ActiveJob] Enqueued ActionMailer::DeliveryJob (Job ID: 7251d23c-0dd4-43c3-a28e-96a1ea96942d) to Async(mailers) with arguments: "Mailer", "issue_add", "deliver_now", #<GlobalID:0x0000000012bef610 @uri=#<URI::GID gid://redmine-app/User/81>>, #<GlobalID:0x0000000012bee7d8 @uri=#<URI::GID gid://redmine-app/Issue/15110>>
[ActiveJob] Enqueued ActionMailer::DeliveryJob (Job ID: 6eb3bbc0-7440-4ac7-8da1-5f15b46899ee) to Async(mailers) with arguments: "Mailer", "issue_add", "deliver_now", #<GlobalID:0x0000000012bcee88 @uri=#<URI::GID gid://redmine-app/User/87>>, #<GlobalID:0x0000000012bce668 @uri=#<URI::GID gid://redmine-app/Issue/15110>>
[ActiveJob] Enqueued ActionMailer::DeliveryJob (Job ID: 95e3cb40-cf2d-4c92-b7d2-d0d848ce97f2) to Async(mailers) with arguments: "Mailer", "issue_add", "deliver_now", #<GlobalID:0x0000000012bbed80 @uri=#<URI::GID gid://redmine-app/User/3>>, #<GlobalID:0x0000000012bbe510 @uri=#<URI::GID gid://redmine-app/Issue/15110>>
Redirected to https://EDITED/redmine/issues/15110
Completed 302 Found in 474ms (ActiveRecord: 136.1ms)
[ActiveJob] [ActionMailer::DeliveryJob] [46277ca8-57a0-4908-a3e6-bf7096e13211] Performing ActionMailer::DeliveryJob (Job ID: 46277ca8-57a0-4908-a3e6-bf7096e13211) from Async(mailers) with arguments: "Mailer", "issue_add", "deliver_now", #<GlobalID:0x0000000012cc1ea8 @uri=#<URI::GID gid://redmine-app/User/1>>, #<GlobalID:0x0000000012cc1458 @uri=#<URI::GID gid://redmine-app/Issue/15110>>
[ActiveJob] [ActionMailer::DeliveryJob] [583bace0-30e7-4b79-8f89-72263fc81527] Performing ActionMailer::DeliveryJob (Job ID: 583bace0-30e7-4b79-8f89-72263fc81527) from Async(mailers) with arguments: "Mailer", "issue_add", "deliver_now", #<GlobalID:0x0000000012be7d98 @uri=#<URI::GID gid://redmine-app/User/16>>, #<GlobalID:0x0000000012be7280 @uri=#<URI::GID gid://redmine-app/Issue/15110>>
[ActiveJob] [ActionMailer::DeliveryJob] [a804b4b4-a0bf-4252-9c2c-8d13a975edc2] Performing ActionMailer::DeliveryJob (Job ID: a804b4b4-a0bf-4252-9c2c-8d13a975edc2) from Async(mailers) with arguments: "Mailer", "issue_add", "deliver_now", #<GlobalID:0x0000000012be4580 @uri=#<URI::GID gid://redmine-app/User/7>>, #<GlobalID:0x0000000012bdf7d8 @uri=#<URI::GID gid://redmine-app/Issue/15110>>
[ActiveJob] [ActionMailer::DeliveryJob] [31e43fc3-d1db-44d1-8ec2-732ded7b209a] Performing ActionMailer::DeliveryJob (Job ID: 31e43fc3-d1db-44d1-8ec2-732ded7b209a) from Async(mailers) with arguments: "Mailer", "issue_add", "deliver_now", #<GlobalID:0x0000000012ca9970 @uri=#<URI::GID gid://redmine-app/User/4>>, #<GlobalID:0x0000000012ca8958 @uri=#<URI::GID gid://redmine-app/Issue/15110>>
Started GET "/redmine/issues/15110" for EDITED at 2019-10-25 08:54:20 +0200
[ActiveJob] [ActionMailer::DeliveryJob] [2135b1ea-73c3-42bb-9e2f-d3f6765bb59c] Performing ActionMailer::DeliveryJob (Job ID: 2135b1ea-73c3-42bb-9e2f-d3f6765bb59c) from Async(mailers) with arguments: "Mailer", "issue_add", "deliver_now", #<GlobalID:0x0000000012b23ab0 @uri=#<URI::GID gid://redmine-app/User/20>>, #<GlobalID:0x0000000012b23290 @uri=#<URI::GID gid://redmine-app/Issue/15110>>

If I leave only 2 members in the project (another testing project) I receive the email notification:

Started PATCH "/redmine/issues/5026" for EDITED at 2019-10-25 08:52:21 +0200
Processing by IssuesController#update as HTML
  Parameters: {"utf8"=>"✓", "authenticity_token"=>"EDITED", "form_update_triggered_by"=>"", "issue"=>{"is_private"=>"0", "project_id"=>"66", "tracker_id"=>"1", "subject"=>"testMail", "description"=>"TestMail", "status_id"=>"10", "priority_id"=>"4", "assigned_to_id"=>"1", "category_id"=>"", "fixed_version_id"=>"", "parent_issue_id"=>"", "start_date"=>"", "due_date"=>"", "estimated_hours"=>"", "done_ratio"=>"0", "custom_field_values"=>{"32"=>[""], "30"=>"0", "31"=>"0", "13"=>"0", "24"=>"", "25"=>"0", "10"=>"100", "9"=>[""], "6"=>"", "8"=>"", "3"=>"", "27"=>"", "26"=>"", "21"=>""}, "notes"=>"test asservatore", "private_notes"=>"0", "lock_version"=>"3"}, "was_default_status"=>"10", "last_journal_id"=>"107355", "commit"=>"Invia", "id"=>"5026"}
  Current user: fabio (id=1)
[ActiveJob] Enqueued ActionMailer::DeliveryJob (Job ID: 2a4ef9a3-5b05-4aee-a553-b0bb44ab52f0) to Async(mailers) with arguments: "Mailer", "issue_edit", "deliver_now", #<GlobalID:0x000000000cf11510 @uri=#<URI::GID gid://redmine-app/User/1>>, #<GlobalID:0x000000000cf10520 @uri=#<URI::GID gid://redmine-app/Journal/107356>>
[ActiveJob] Enqueued ActionMailer::DeliveryJob (Job ID: 49fb2644-e481-4974-a2d9-9fb3da75b9e4) to Async(mailers) with arguments: "Mailer", "issue_edit", "deliver_now", #<GlobalID:0x000000000cefad10 @uri=#<URI::GID gid://redmine-app/User/3>>, #<GlobalID:0x000000000cefa270 @uri=#<URI::GID gid://redmine-app/Journal/107356>>
Redirected to https://EDITED/redmine/issues/5026
Completed 302 Found in 323ms (ActiveRecord: 166.3ms)
[ActiveJob] [ActionMailer::DeliveryJob] [2a4ef9a3-5b05-4aee-a553-b0bb44ab52f0] Performing ActionMailer::DeliveryJob (Job ID: 2a4ef9a3-5b05-4aee-a553-b0bb44ab52f0) from Async(mailers) with arguments: "Mailer", "issue_edit", "deliver_now", #<GlobalID:0x000000000ceb1840 @uri=#<URI::GID gid://redmine-app/User/1>>, #<GlobalID:0x000000000ceb0cb0 @uri=#<URI::GID gid://redmine-app/Journal/107356>>
[ActiveJob] [ActionMailer::DeliveryJob] [49fb2644-e481-4974-a2d9-9fb3da75b9e4] Performing ActionMailer::DeliveryJob (Job ID: 49fb2644-e481-4974-a2d9-9fb3da75b9e4) from Async(mailers) with arguments: "Mailer", "issue_edit", "deliver_now", #<GlobalID:0x000000000cdb4708 @uri=#<URI::GID gid://redmine-app/User/3>>, #<GlobalID:0x000000000cdaf9d8 @uri=#<URI::GID gid://redmine-app/Journal/107356>>

I have updated the previous message, because, after further investigation it seem that the magic number is 30.

To restore the notification function, I have deleted all members in my project and then re-added them. When I deleted members from project I have seen that some users are in multiple groups and also added alone with a role of one of the Group already added.
For example, I have seen the user A that is in developer Group. When I have deleted the developer Group, I have again an occurrence of A as developer (this because in the past, I have added before A as developer and only after I have created a Group of developer with A that I have added to project).

I'm experiencing the same issue also in 4.1 release. There Is an option to set a verbose log to see why some email is enqueued but not sended?
Happy new year!

Hi.
I can confirm that my issue is a regression probably caused by #26791. I have make a workaround replacing deliver_later with deliver in app/models/mailer.rb
I have degraded performance when save a ticket but all notifications are sended!

The subject of this ticket should be changed because the notification not works at all and not only when Only for things I watch or I'm involed in is set.
I'm not a ruby developer or a ruby expert, but it seem that my scenario is the same described here: usually every ticket involve 20 emails notifications to send. After redmine 4.1.0 upgrade, (randomly) only 5-13 are sended but never all 20. It seems that the queue at some point is cancelled and the sending is not completed.
As I have write before, now I use deliver_now (I have replaced deliver because is deprecated) instead of deliver_later: the performance isn't acceptable because to save a single ticket need around 40" (to send 20 email and save changes).

Sorry for my silence on this issue, I’ll try to reproduce the problem based on your last notes.

I presume you're using an in-memory queue which might be an issue here.

FWIW: Jean-Philippe Lang wrote in Redmine 4.0.0, 3.4.7 and 3.3.9 released:

Email delivery now relies on Rails ActiveJob. Emails are sent asynchronously by default. But you should consider configuring a persistent backend for ActiveJob since the default uses an in-memory queue that is not well suited for production environnements:
https://guides.rubyonrails.org/v5.2/active_job_basics.html#job-execution

I've tested now on my local environment and all 28 expected mails are enqueued and performed using the default AsyncAdapter. My test was quite simple, I've added 24 new users as watchers to an issue and I added a note (according to note#13, the problem is not longer related to the groups).

Mischa The Evil wrote:

I presume you're using an in-memory queue which might be an issue here.

Yes, this is my assumption as well, but I'm curios why the queue is canceled after some emails are sent. Anyway, I would recommend to configure sidekiq or another queuing backend (https://edgeguides.rubyonrails.org/active_job_basics.html#starting-the-backend) and if the issue still persists, we continue from there.

Mischa The Evil wrote:

I presume you're using an in-memory queue which might be an issue here.

FWIW: Jean-Philippe Lang wrote in Redmine 4.0.0, 3.4.7 and 3.3.9 released:

Email delivery now relies on Rails ActiveJob. Emails are sent asynchronously by default. But you should consider configuring a persistent backend for ActiveJob since the default uses an in-memory queue that is not well suited for production environnements:
https://guides.rubyonrails.org/v5.2/active_job_basics.html#job-execution

yep, I have totally missed this advice. I try with sidekiq as Marius comment and return here ASAP.
One question: because redmine Is intended for production Pourpose, should be the case to provide a bundle sidekiq or similar with a minimal configuration? Thanks

pasquale [:dedalus] wrote:

yep, I have totally missed this advice. I try with sidekiq as Marius comment and return here ASAP.

Great, waiting for your results.

One question: because redmine Is intended for production Pourpose, should be the case to provide a bundle sidekiq or similar with a minimal configuration? Thanks

Good idea. I think we can improve at least the documentation.

Marius BALTEANU wrote:

pasquale [:dedalus] wrote:

yep, I have totally missed this advice. I try with sidekiq as Marius comment and return here ASAP.

Great, waiting for your results.

Using Sidekiq all works fine.
Thanks, this issue can be closed as resolved.

Marius BALTEANU wrote:

pasquale [:dedalus] wrote:

yep, I have totally missed this advice. I try with sidekiq as Marius comment and return here ASAP.

Great, waiting for your results.

One question: because redmine Is intended for production Pourpose, should be the case to provide a bundle sidekiq or similar with a minimal configuration? Thanks

Good idea. I think we can improve at least the documentation.

Pasquale, I've added a wiki page with the instructions, please take a look and let us know how do you think. This is the issue #33589 and this is the page SidekiqConfiguration.

I don't think this is an ideal solution. Sidekiq depends on redis. Why would we need a cache server for sending email? The way I see it, we have three options for making this work for users out the box:

1. Switch to the Inline adapter. This might not be ideal for various reasons;
2. Depend on sucker_punch. Its ActiveJob adapter is supposed to work the same as the Async adapter, which makes sense since the both use concurrent_ruby. This might not solve the problem;
3. Depend on delayed_job using the data_mapper backend. This requires a few more Ruby gems as dependencies, but it might be the least painful for users when it comes to deployment.

We're testing all three of these this week. In the production log, we see at most five emails being sent although many more were queued. These disappear since the job finishes before the rest of them can be sent, which is how the Async queue backend works.

Testing the inline adaptor was as expected. An update which required sending mails to nine people took 43s to execute. At least all mails got delivered. However, this is not viable in production.

Testing the sucker_punch adaptor exceeded predictions. It's incredibly fast and all emails get delivered. I recommend Redmine depend on it and include it as part of the default configuration. It's not going to be a good UX moving forward if people have to dig through documentation to discover why their mails are disappearing into the void.

For the send_reminders rake job to work properly, we had to set:

SuckerPunch.shutdown_timeout = 180

in config/additional_environment.rb. Else, the job would hit its short default timeout of 8s before completing.

Future Link Corporation pkgsrc team wrote:

Testing the sucker_punch adaptor exceeded predictions. It's incredibly fast and all emails get delivered. I recommend Redmine depend on it and include it as part of the default configuration. It's not going to be a good UX moving forward if people have to dig through documentation to discover why their mails are disappearing into the void.

Following your recommendation we installed Sucker Punch and it's working flawlessly. Many thanks!

Guillermo ML wrote:

Following your recommendation we installed Sucker Punch and it's working flawlessly. Many thanks!

Could you please share what have you done to switch mail processing to SuckerPunch?

Petr Bystrov wrote:

Could you please share what have you done to switch mail processing to SuckerPunch?

1. Copy into folder /config the example file, renaming it:
   

   cp config/additional_environment.rb.example config/additional_environment.rb

2. Add this line into the file:
   

   config.active_job.queue_adapter = :sucker_punch

3. Edit Gemfile and add gem:
   

   gem 'sucker_punch'

4. Install gem
   

   bundle

5. Reboot server.

Having said that we finally remove the gem. We found that our problems where not related to ActiveJob, and we decided to go ahead with it.
Both worked flawlessly but in the end were similar (memory only queues) so we stick with ActiveJob. Our stage server with 2 nodes (2GB and 2vCPU) sent 990 mails successfully in 24s.