Patch #15785
closed
Support more character encodings in incoming emails
0%
Description
The MailHandler currently tries to convert the encoding of the body of incoming emails to UTF-8 by directly passing the charset of the email body to Redmine::CodesetUtil.to_utf8, see source:/trunk/app/models/mail_handler.rb@12473#L413. Redmine::CodesetUtil.to_utf8 in turn only handles encodings known to Ruby (on 1.9+), which might not be the case for the charset of an incoming email.
Planio recently had the case of a user sending emails in Korean with the charset ks_c_5601-1987, which Ruby doesn't know about. This can be mitigated thanks to Mail::RubyVer.pick_encoding which will try to pick a Ruby encoding compatible with the charset of the email (Mail::RubyVer is a wrapper for the correct functions for the current Ruby version).
I will post a patch for this shortly.
Files
Related issues
Updated by Felix Schäfer over 10 years ago
- File 15785.patch 15785.patch added
Here is the patch with a test.
Updated by Jan from Planio www.plan.io over 10 years ago
- Status changed from New to Confirmed
- Target version set to Candidate for next minor release
Updated by Jan from Planio www.plan.io over 10 years ago
- Tracker changed from Defect to Patch
Updated by Toshi MARUYAMA over 10 years ago
This patch looks good.
But, Korean encoding detection is in mail 2.5.4.
https://github.com/mikel/mail/commit/88457e66ec5b298bd198b5c3b6739f05320a16ee
Many users posted that mail 2.5.4 has trouble on redmine.org.
For test passes, we need to restrict mail 2.5.4 in Gemfile.
Updated by Toshi MARUYAMA over 10 years ago
Toshi MARUYAMA wrote:
This patch looks good.
But, Korean encoding detection is in mail 2.5.4.
https://github.com/mikel/mail/commit/88457e66ec5b298bd198b5c3b6739f05320a16ee
This means in mail 2.5.4, not in mail 2.5.3.
Updated by Toshi MARUYAMA over 10 years ago
It seems Mail::VERSION returns mail version.
https://github.com/mikel/mail/blob/2-5-stable/lib/mail/version.rb
Updated by Felix Schäfer over 10 years ago
Mail::VERSION.version returns the version of the Mail gem indeed.
I know korean would only work with Mail 2.5.4, and we don't mind requiring it in the Planio Gemfiles to make sure it is supported for us. I think the patch is useful even with earlier versions of the Mail gem, as Mail::RubyVer.pick_encoding is available in earlier versions too and still adds more supported encodings. I agree with you that the test I've written would require Mail 2.5.4 though, you are right.
Should I write a new test that makes sure that the patch works but doesn't require 2.5.4?
Furthermore, what type of errors do users encounter with Mail 2.5.4?
Updated by Toshi MARUYAMA over 10 years ago
- Subject changed from Support more charsets in incoming emails to Support more charctor encoding in incoming emails
Updated by Toshi MARUYAMA over 10 years ago
- Subject changed from Support more charctor encoding in incoming emails to Support more character encoding in incoming emails
Updated by Toshi MARUYAMA over 10 years ago
- Subject changed from Support more character encoding in incoming emails to Support more character encodings in incoming emails
Updated by Toshi MARUYAMA over 10 years ago
Felix Schäfer wrote:
Furthermore, what type of errors do users encounter with Mail 2.5.4?
See #13698.
But Rails 3.2.16 requires mail 2.5.4 or higher.
https://github.com/rails/rails/blame/v3.2.16/actionmailer/actionmailer.gemspec#L23
Updated by Toshi MARUYAMA over 10 years ago
- Target version changed from Candidate for next minor release to 2.5.0
Updated by Toshi MARUYAMA over 10 years ago
- Status changed from Confirmed to Closed
Committed in trunk r12474 and tests pass, thanks.
Updated by Anton Nepomnyaschih over 10 years ago
Is it duplicate for #14675 ? Can we update our Redmine and get the Defect fixed? =)
Updated by Toshi MARUYAMA about 10 years ago
- Has duplicate Defect #14675: redmine:email:receive_imap failed with localizaed characters in mail body added
Updated by Toshi MARUYAMA about 10 years ago
Anton Nepomnyaschih wrote:
Is it duplicate for #14675 ? Can we update our Redmine and get the Defect fixed? =)
Thank you for your pointing.
Updated by Toshi MARUYAMA about 10 years ago
- Has duplicate deleted (Defect #14675: redmine:email:receive_imap failed with localizaed characters in mail body)
Updated by Toshi MARUYAMA over 9 years ago
- Related to Patch #18047: MailHandler: Don't use String#respond_to?(:force_encoding) to differentiate between Ruby 1.8 and Ruby 1.9 added