Project

General

Profile

Actions

Defect #16122

open

redmine:email:receive should not abort with buggy mail

Added by Vedran Rodic about 10 years ago. Updated about 9 years ago.

Status:
New
Priority:
Normal
Assignee:
-
Category:
Email receiving
Target version:
-
Start date:
Due date:
% Done:

0%

Estimated time:
Resolution:
Affected version:

Description

Hi

I'm testing issue creation via email with redmine-2.4.3. It works fine when I have plain Ascii mails, but fails with Croatian characters. Gmail chooses to use latin2 for them, and that works fine in all other apps, but not with redmine 2.4.3 when receiving email. I've attached the .eml file that gmail sends to our mail server.

I've also tried with patch in http://www.redmine.org/projects/redmine/repository/revisions/12474, but issue is still present.

rake -f ~/redmine-2.4.3/Rakefile redmine:email:receive_imap RAILS_ENV="production" host=xxxxxxx username=xxxxx password=xxxx project=operativa tracker=Bug allow_override=tracker,priority

rake aborted!
"\xB9" from ASCII-8BIT to UTF-8
/home/redmine/.rvm/gems/ruby-1.9.3-p484/gems/mail-2.5.4/lib/mail/fields/unstructured_field.rb:149:in `encode!'
/home/redmine/.rvm/gems/ruby-1.9.3-p484/gems/mail-2.5.4/lib/mail/fields/unstructured_field.rb:149:in `fold'
/home/redmine/.rvm/gems/ruby-1.9.3-p484/gems/mail-2.5.4/lib/mail/fields/unstructured_field.rb:103:in `wrapped_value'
/home/redmine/.rvm/gems/ruby-1.9.3-p484/gems/mail-2.5.4/lib/mail/fields/unstructured_field.rb:69:in `do_encode'
/home/redmine/.rvm/gems/ruby-1.9.3-p484/gems/mail-2.5.4/lib/mail/fields/unstructured_field.rb:51:in `encoded'
/home/redmine/.rvm/gems/ruby-1.9.3-p484/gems/mail-2.5.4/lib/mail/field.rb:167:in `method_missing'
/home/redmine/.rvm/gems/ruby-1.9.3-p484/gems/mail-2.5.4/lib/mail/header.rb:206:in `block in encoded'
/home/redmine/.rvm/gems/ruby-1.9.3-p484/gems/mail-2.5.4/lib/mail/header.rb:205:in `each'
/home/redmine/.rvm/gems/ruby-1.9.3-p484/gems/mail-2.5.4/lib/mail/header.rb:205:in `encoded'
/home/redmine/.rvm/gems/ruby-1.9.3-p484/gems/mail-2.5.4/lib/mail/message.rb:1801:in `encoded'
/home/redmine/.rvm/gems/ruby-1.9.3-p484/gems/actionmailer-3.2.16/lib/action_mailer/base.rb:434:in `set_payload_for_mail'
/home/redmine/.rvm/gems/ruby-1.9.3-p484/gems/actionmailer-3.2.16/lib/action_mailer/base.rb:403:in `block in receive'
/home/redmine/.rvm/gems/ruby-1.9.3-p484/gems/activesupport-3.2.16/lib/active_support/notifications.rb:123:in `block in instrument'
/home/redmine/.rvm/gems/ruby-1.9.3-p484/gems/activesupport-3.2.16/lib/active_support/notifications/instrumenter.rb:20:in `instrument'
/home/redmine/.rvm/gems/ruby-1.9.3-p484/gems/activesupport-3.2.16/lib/active_support/notifications.rb:123:in `instrument'
/home/redmine/.rvm/gems/ruby-1.9.3-p484/gems/actionmailer-3.2.16/lib/action_mailer/base.rb:401:in `receive'
/home/redmine/redmine-2.4.3/app/models/mail_handler.rb:46:in `receive'
/home/redmine/redmine-2.4.3/lib/redmine/imap.rb:35:in `block in check'
/home/redmine/redmine-2.4.3/lib/redmine/imap.rb:32:in `each'
/home/redmine/redmine-2.4.3/lib/redmine/imap.rb:32:in `check'
/home/redmine/redmine-2.4.3/lib/tasks/email.rake:125:in `block (3 levels) in <top (required)>'
/home/redmine/.rvm/gems/ruby-1.9.3-p484/bin/ruby_executable_hooks:15:in `eval'
/home/redmine/.rvm/gems/ruby-1.9.3-p484/bin/ruby_executable_hooks:15:in `<main>'
Tasks: TOP => redmine:email:receive_imap
(See full trace by running task with --trace)

Please tell if you need more data or testing.

Issue looks very similar as the one here: http://stackoverflow.com/questions/19450199/redmine-receive-mail-field


Related issues

Related to Redmine - Patch #29614: redmine:email:read and rdm-mailhandler.rb should use safe_receive instead of receiveClosedJean-Philippe Lang

Actions
Has duplicate Redmine - Defect #16261: tasks by emailClosed

Actions
Has duplicate Redmine - Defect #18649: mail_handler.rb - does not encode non-utf8 charactersClosed

Actions
Actions #1

Updated by Toshi MARUYAMA about 10 years ago

  • File deleted (gmail-iso8859-2.eml)
Actions #2

Updated by Toshi MARUYAMA about 10 years ago

  • Category set to Email receiving
Actions #3

Updated by Toshi MARUYAMA about 10 years ago

I removed attachment because it includes private data.

Actions #4

Updated by Toshi MARUYAMA about 10 years ago

  • Status changed from New to Needs feedback
  • Resolution set to Duplicate

I added test and test passes.

Actions #5

Updated by Vedran Rodic about 10 years ago

When trying with my original file it doesn't pass, but that might be a test infrastructure problem.

Your file passes the test, so I guess the test isn't complete. I'll try to investigate by printing out what we get from our email system but I'm a ruby newbie and would like to have some advice on how to proceed.

Btw, this is the output when running with my original .eml file, how to get a full trace of the failure instead of just this:
~/redmine-2.4.3]# ruby test/unit/mail_handler_test.rb -n test_gmail_iso8859_2
Run options: -n test_gmail_iso8859_2

  1. Running tests:

F

Finished tests in 3.561271s, 0.2808 tests/s, 0.2808 assertions/s.

1) Failure:
test_gmail_iso8859_2(MailHandlerTest) [test/unit/mail_handler_test.rb:535]:
Expected false to be a kind of Issue, not FalseClass.

1 tests, 1 assertions, 1 failures, 0 errors, 0 skips

Actions #6

Updated by Vedran Rodic about 10 years ago

Okay, my original report is not complete.

I added puts(msg) in imap.rb to see the complete message we get get from our system and it contained this (containing part of my signature):

X-Ham-Report: Spam detection software, running on the system "xxxxxxx", has
identified this incoming email as possible spam. The original message
has been attached to this so you can view it (if it isn't spam) or label
similar future email. If you have any questions, see
root\@localhost for details.

Content preview:  -- Vedran Rodi� Software developer Tau on-line d.o.o. Hondlova
2/11, HR-10000 Zagreb Phone+385 (1) xxxxxxx [...]
X-Spam-Flag: NO

Now, this � is not in any encoding mentioned anywhere.

This is added by our SpamAssassin setup, which might be buggy.

I have a feeling that redmine shouldn't just abort processing this message in these cases, but it's your choice on how to handle this.

Actions #7

Updated by Toshi MARUYAMA about 10 years ago

  • Subject changed from error with redmine:email:receive when receiving ISO-8859-2 mails mailed from gmail to redmine:email:receive should not abort with buggy mail
  • Status changed from Needs feedback to New
  • Resolution deleted (Duplicate)
Actions #8

Updated by Toshi MARUYAMA about 10 years ago

Actions #9

Updated by Andre Oliveira over 9 years ago

I solved that problem in my company by forcing the encoding...

In redmine_folder/app/models/mail_handler.rb

add:
email.force_encoding('UTF-8')

after:
def self.receive(email, options={})

and remove:
email.force_encoding('ASCII-8BIT') if email.respond_to?(:force_encoding)

Hope it helps

Actions #10

Updated by Toshi MARUYAMA over 9 years ago

Luis Fontes wrote:

I solved that problem in my company by forcing the encoding...

In redmine_folder/app/models/mail_handler.rb

add:
email.force_encoding('UTF-8')

Could you try this instead?

email.force_encoding('ASCII-8BIT')

Because Rails 4 porting has this code.
source:trunk/app/models/mail_handler.rb@13623#L45

after
def self.receive(email, options={})

Actions #11

Updated by Toshi MARUYAMA over 9 years ago

  • Has duplicate Defect #18649: mail_handler.rb - does not encode non-utf8 characters added
Actions #12

Updated by Toshi MARUYAMA over 9 years ago

FTR:
In #18649, there is request that Redmine should not ignore but replace invalid characters.

Actions #13

Updated by Andre Oliveira about 9 years ago

Toshi MARUYAMA wrote:

Could you try this instead?

email.force_encoding('ASCII-8BIT')

Because Rails 4 porting has this code.
source:trunk/app/models/mail_handler.rb@13623#L45

That did not work in my case,I forgot to tell to remove that line.

When I did this changes emails without subject were causing errors too, because an Issue needs to have the subject, so if anyone has this problem too just need to add

issue.subject = '(no subject)' if issue.subject.blank?

after
issue.subject = cleaned_up_subject

(this is in the receive_issue method)

Actions #14

Updated by Go MAEDA over 5 years ago

  • Related to Patch #29614: redmine:email:read and rdm-mailhandler.rb should use safe_receive instead of receive added
Actions

Also available in: Atom PDF