Patch #36691

Background job and dedicated status for project deletion

Added by Jens Krämer 3 months ago. Updated 10 days ago.

Status:ResolvedStart date:
Priority:NormalDue date:
Assignee:Marius BALTEANU% Done:

0%

Category:Administration
Target version:5.1.0

Description

Due to the deletion of dependent objects (issues etc) and child projects,
project deletion may take a long time.

This patch, which was extracted from [Planio](https://plan.io/redmine-hosting),
moves the actual project deletion into an ActiveJob job. It also introduces a
new project status (SCHEDULED_FOR_DELETION) that is used to effectively hide
the project that is about to be deleted (and any potential descendant projects)
from the system immediately.

A security notification is sent out to the user who deleted the project,
informing about success / failure.

The projects list is extended to be able to filter for the new status,
so in case of a failure, the project can still be accessed for
examination.

0001-background-job-for-project-deletion.patch Magnifier (14 KB) Jens Krämer, 2022-02-23 10:14

fix_failing_tests.diff Magnifier (2.8 KB) Jens Krämer, 2022-04-06 08:18


Related issues

Related to Redmine - Patch #31076: Issues CSV / PDF export via ActiveJob New
Related to Redmine - Feature #33422: Re-implement admin project list using ProjectQuery system Closed

Associated revisions

Revision 21591
Added by Marius BALTEANU 10 days ago

Background job for project deletion (#36691).

Due to the deletion of dependent objects (issues etc), project deletion may take a long time.

This patch moves the actual project deletion into an ActiveJob job. It also introduces a new project status (SCHEDULED_FOR_DELETION) that is used to effectively hide the project that is about to be deleted (and any potential descendant projects) from the system immediately.

A security notification is sent out to the user that deleted the project, informing about success / failure.

The projects list is extended to be able to filter for the new status, so in case of a failure, the project can still be accessed for examination.

Patch by Jens Krämer.

Revision 21592
Added by Marius BALTEANU 10 days ago

Adds projects bulk delete (#36691).

Patch by Jens Krämer.

Revision 21593
Added by Marius BALTEANU 10 days ago

Fix random failing tests (#36691).

Patch by Jens Krämer.

Revision 21598
Added by Marius BALTEANU 8 days ago

Remove test code (#36691).

Revision 21602
Added by Go MAEDA 2 days ago

Update locales (#36691).

History

#1 Updated by Marius BALTEANU 3 months ago

  • Target version set to Candidate for next major release

Nice feature!

#2 Updated by Marius BALTEANU 3 months ago

  • Related to Patch #31076: Issues CSV / PDF export via ActiveJob added

#3 Updated by Marius BALTEANU 3 months ago

Jens, should we take into consideration also #34987?

#4 Updated by James H 3 months ago

maybe my group members issue #36696 can benefit from this as well

#5 Updated by Jens Krämer 3 months ago

Marius BALTEANU wrote:

Jens, should we take into consideration also #34987?

Ah, that's interesting. If I understand correctly, that would mean the project record (plus potential child projects' records) would be deleted synchronously, but removal of all dependent objects would be pushed to the job queue.

I see few problems with that:

1. Data integrity. Before the job is finished, we have an inconsistent database (i.e. issues referencing a nonexisting project). I am not sure we handle such a situation gracefully (i.e. resulting in an HTTP 404 vs a 500) in all cases.
2. Two transactions instead of one. Currently, if the project deletion fails due to a problem somewhere along the way (due to a failing callback for example), the transaction is rolled back and everything is as if nothing happened. Splitting the process in two separate transactions (one for the project, one later for the dependent objects), means the project record will be gone even if some dependent object insists on not being destroyed. I do not think core has any such callbacks, but plugins might.
3. Does not work at all when there are foreign keys referencing `projects.id`. We do not have such in core, but again, there may be plugins that do.

#6 Updated by Marius BALTEANU 3 months ago

Thanks Jens for your quick response.

#7 Updated by Marius BALTEANU about 1 month ago

  • Target version changed from Candidate for next major release to 5.1.0

#8 Updated by Jens Krämer about 1 month ago

Marius, I just have re-submitted this patch as part of a larger work on the admin projects list in #33422. It would be great if you could have a look and consider integrating that one instead.

#9 Updated by Marius BALTEANU about 1 month ago

There is a random failing test, you can reproduce by running the tests using the following command: TESTOPTS="--seed=42328" rake test

Failure:
DestroyProjectJobTest#test_should_destroy_project_and_send_email [/work/test/unit/jobs/destroy_project_job_test.rb:60]:
No enqueued job found with {:job=>ActionMailer::MailDeliveryJob, :args=>#<Proc:0x0000aaaafe3aec98 /work/test/unit/jobs/destroy_project_job_test.rb:62 (lambda)>}

Jens, can you take a look, please?

#10 Updated by Marius BALTEANU about 1 month ago

  • Related to Feature #33422: Re-implement admin project list using ProjectQuery system added

#11 Updated by Marius BALTEANU about 1 month ago

  • Assignee set to Jens Krämer

#12 Updated by Jens Krämer about 1 month ago

this is really strange, looks like under some circumstances emails are processed inline and not result in a queued mailer job. I did not find the cause of this, but the attached patch would handle both cases in the tests (the same issue exists in the test case for the job that handles deletion of multiple projects).

#13 Updated by Marius BALTEANU 10 days ago

  • Status changed from New to Resolved
  • Assignee changed from Jens Krämer to Marius BALTEANU

Both patches from #33422 and the fix for tests committed, thank you for your contribution, really useful feature.

Also available in: Atom PDF