Project

General

Profile

Actions
Copy link

Defect #27691

closed

Issue created from mail_handler with description empty from

Added by Julien TEHERY almost 7 years ago. Updated about 6 years ago.

Status:
Closed
Priority:
Normal
Assignee:
-
Category:
Email receiving
Target version:
-
Start date:
Due date:
% Done:

0%

Estimated time:
Resolution:
Fixed
Affected version:
3.2.8

Description

Env : redmine 3.2.8 - PRODUCTION
OS: Debian 8.9
Mailhandler: source:tags/3.2.8/extra/mail_handler/rdm-mailhandler.rb

Migrating our production redmine from 3.2.1 to 3.2.8 , we noticed that some emails with french sepcial characters were creating empty issues.

Here are the details:

- mail is sent to
- on mailserver, we use latest version of rdm-mailhandler.rb which forwards email to our redmine in specific projects.
- on redmine server, we see both succeeding emails and some others who are not correctly transformed into issues.

On production.log, we don't see any particular problem
Here is an example of an email becoming an empty issue:

Started POST "/mail_handler" for 10.123.1.3 at 2017-12-01 11:49:52 +0100
Processing by MailHandlerController#index as HTML
  Parameters: {"key"=>"XIQlZmuj8wzsyWBC4ucB", "email"=>"From julinux@mycompany.fr  Fri Dec  1 11:49:52 2017\nReturn-Path: <julinux@mycompany.fr>\nX-Original-To: ticket@mycompany.fr\nDelivered-To: ticket@mycompany.fr\nReceived: from localhost (unknown [127.0.0.1])\n\tby mail.mycompany.fr (Postfix::smtpd) with ESMTP id 5E1A7559\n\tfor <ticket@mycompany.fr>; Fri,  1 Dec 2017 11:49:52 +0100 (CET)\nDKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=mycompany.fr;\n\ts=mycompany.fr; t=1512125392;\n\tbh=+cEL8GCaNKPGfAl6U7yCNuKL4UTN2UkhX+jOBsmT7iY=;\n\th=To:From:Subject:Date:From;\n\tb=QuPhO2+UuQM8K5Bc8AMNRXC/bC7pTaXq42ambTZz89BZcyzAFKiSZzLdAr99T+o/o\n\t thGNDzH26zgG4oxQK3szqxKzsJgsXijhJPftgDECFWIb5HOiCnb77hbYvBJ5dQ1ECz\n\t BlyasjpiMHrsALRjNAEettV58VMlDhqQaj8J7rSc=\nDKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=mycompany.fr;\n\ts=mycompany.fr; t=1512125392;\n\tbh=+cEL8GCaNKPGfAl6U7yCNuKL4UTN2UkhX+jOBsmT7iY=;\n\th=To:From:Subject:Date:From;\n\tb=QuPhO2+UuQM8K5Bc8AMNRXC/bC7pTaXq42ambTZz89BZcyzAFKiSZzLdAr99T+o/o\n\t thGNDzH26zgG4oxQK3szqxKzsJgsXijhJPftgDECFWIb5HOiCnb77hbYvBJ5dQ1ECz\n\t BlyasjpiMHrsALRjNAEettV58VMlDhqQaj8J7rSc=\nX-Virus-Scanned: Debian amavisd-new at debian8amd64prodOE\nX-Spam-Flag: NO\nX-Spam-Score: -2.998\nX-Spam-Level:\nX-Spam-Status: No, score=-2.998 tagged_above=-999 required=5.5\n\ttests=[ALL_TRUSTED=-1, BAYES_00=-1.9, DKIM_SIGNED=0.1,\n\tDKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HTML_MESSAGE=0.001,\n\tURIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no\nReceived: from mail.mycompany.fr ([127.0.0.1])\n\tby localhost (mail.mycompany.fr [127.0.0.1]) (amavisd-new, port 10024)\n\twith LMTP id c9qer7n5h2LZ for <ticket@mycompany.fr>;\n\tFri,  1 Dec 2017 11:49:51 +0100 (CET)\nReceived: from [192.168.103.194] (unknown [109.190.128.27])\n\t(using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits))\n\t(No client certificate requested)\n\tby mail.mycompany.fr (Postfix::smtpd) with ESMTPSA id 994FA4D1\n\tfor <ticket@mycompany.fr>; Fri,  1 Dec 2017 11:49:51 +0100 (CET)\nDKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=mycompany.fr;\n\ts=mycompany.fr; t=1512125391;\n\tbh=+cEL8GCaNKPGfAl6U7yCNuKL4UTN2UkhX+jOBsmT7iY=;\n\th=To:From:Subject:Date:From;\n\tb=TMesqjQtZ7ut1zYR91GtKcyz6GxYHjO7zoV6tKtAlGlT1K7EQlCC+LX0NVKGBWkgp\n\t mP4FjKVN+W1lTvgeHxFd1zcHDISeC0WGqHv5S5D94PEdaj8bpbluS0IeROPxgo8fhH\n\t IRR0nOqnWXRi15ck3qm7WR61kg/0I02o4MpQSryQ=\nDKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=mycompany.fr;\n\ts=mycompany.fr; t=1512125391;\n\tbh=+cEL8GCaNKPGfAl6U7yCNuKL4UTN2UkhX+jOBsmT7iY=;\n\th=To:From:Subject:Date:From;\n\tb=TMesqjQtZ7ut1zYR91GtKcyz6GxYHjO7zoV6tKtAlGlT1K7EQlCC+LX0NVKGBWkgp\n\t mP4FjKVN+W1lTvgeHxFd1zcHDISeC0WGqHv5S5D94PEdaj8bpbluS0IeROPxgo8fhH\n\t IRR0nOqnWXRi15ck3qm7WR61kg/0I02o4MpQSryQ=\nTo: ticket@mycompany.fr\nFrom: Julien TEHERY <julinux@mycompany.fr>\nSubject: TEST 16\nMessage-ID: <e51fd0a1-b014-ef78-2ff2-4ef38a130388@mycompany.fr>\nDate: Fri, 1 Dec 2017 11:49:51 +0100\nUser-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101\n Thunderbird/52.4.0\nMIME-Version: 1.0\nContent-Type: multipart/alternative;\n boundary=\"------------4858C4CCDE564230BCC36310\"\nContent-Language: fr\n\nThis is a multi-part message in MIME format.\n--------------4858C4CCDE564230BCC36310\nContent-Type: text/plain; charset=utf-8; format=flowed\nContent-Transfer-Encoding: 8bit\n\nTEST 16\n\n-- \nCordialement,\n\nJulien Téhéry - Ingénieur Systèmes et Réseaux\nSociété mycompany - www.mycompany.ff
r <http://www.mycompany.fr>\nTél Support : 05 49 62 26 07\n\n\n--------------4858C4CCDE564230BCC36310\nContent-Type: text/html; charset=utf-8\nContent-Transfer-Encodii
ng: 8bit\n\n<html>\n  <head>\n\n    <meta http-equiv=\"content-type\" content=\"text/html; charset=utf-8\">\n  </head>\n  <body text=\"#000000\" bgcolor=\"#FFFFFF\">\n    <p>TEST 16<br>\n    </p>\n    <div class=\"moz-signature\">-- <br>\n      Cordialement,<br>\n      <br>\n      Julien Téhéry - Ingénieur Systèmes et Réseaux<br>>
\n      Société mycompany - <a href=\"http://www.mycompany.fr\"\n        title=\"mycompany\" target=\"_blank\">www.mycompany.fr</a><br>\n      Tél Suppoo
rt : 05 49 62 26 07<br>\n      <br>\n    </div>\n  </body>\n</html>\n\n--------------4858C4CCDE564230BCC36310--\n", "allow_override"=>"project,tracker,priority,status", "unknown_user"=>"accept", "default_group"=>nil, "no_account_notice"=>nil, "no_notification"=>nil, "no_permission_check"=>"1", "project_from_subaddress"=>nil, "issue"=>{"project"=>"clients-mycompany", "tracker"=>"Support"}}
  Rendered queries/_filters.html.erb (12.5ms)
  Rendered queries/_columns.html.erb (1.8ms)
  Rendered issues/_list.html.erb (35.5ms)
  Rendered issues/_sidebar.html.erb (21.2ms)
  Rendered issues/index.html.erb within layouts/base (81.4ms)
Completed 200 OK in 566ms (Views: 132.4ms | ActiveRecord: 12.4ms)

So I tried to figure out what was different in the incoming emails, looking at the source of them.
Didn't find anything particular excepted that "good emails" had "Content-Transfer-Encoding: quoted-printable" in their source code.

I don't know if this can be related or not to my problem.
But the fact is we did'nt changed anything regarding mail_handler or mailserver, and we lost the ability to get emails from everyone.
I must precise that we allready used latest version of mail_handler (see links above) before upgrading.

Any help would be appreciated !

Files

27691-test.patch (1.68 KB) 27691-test.patch Yuichi HARADA, 2018-10-15 09:20

Related issues

Related to Redmine - Defect #29365: MailHandlerTest#test_add_issue_with_localized_attributes fails with mail gem 2.7.0Closed

Actions
Related to Redmine - Feature #29443: Update mail gem (~> 2.7.1)ClosedGo MAEDA

Actions
Actions
Copy link
 #1

Updated by Julien TEHERY almost 7 years ago

Here are some precisions about the incoming emails:

Looking at the source code of the emails, here is what i noticed.

  • BAD INCOMING MAIL TURNED INTO EMPTY ISSUE
    Content-Type: text/plain; charset=utf-8; format=flowed
    Content-Transfer-Encoding: 8bit
  • GOOD MAIL TURNED INTO FULLFILLED ISSUE
    Content-Type: text/plain; charset=UTF-8
    Content-Transfer-Encoding: quoted-printable

I really don't have any idea why 8bit encoding would cause such a behavior, but the fact is that all
bad emails are formated this way.

Actions
Copy link
 #2

Updated by Julien TEHERY almost 7 years ago

Ok, i can confirm this is the cause.

I'm using thunderbird to send emails, and it's configured to transfer mails in 8bits by default.
It has always been the case and worked until the upgrade in 3.2.8.
Now i had to setup thunedrbird to use MIME for outgoing email. With that email arrive with "Content-Transfer-Encoding: quoted-printable" and are well processed by the mail_handler.

I think it's clearly a bug, because it has always worked through the past. I won't tell my customers to change their MTA configuration to send me emails.

If it can help someone...

Actions
Copy link
 #3

Updated by Jessie Lee almost 7 years ago

encountering same parsing bug with extended utf-8 characters. same discovery as comment 1.

When set to unknown_user=create we get 

MailHandler: ignoring email from unknown user []
Completed 422 Unprocessable Entity in 4ms (ActiveRecord: 0.0ms)

and when set to unknown_user=accept we get a blank no subject no description email.

example of source of email that causes problem below:

From: REDACTED
Subject: emdash test
To: REDACTED
Message-ID: <eba059c5-7557-9744-61e2-8c52c46cea86@palantetech.coop>
Date: Tue, 5 Dec 2017 13:33:28 -0500
User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:52.0) Gecko/20100101
 Thunderbird/52.5.0
MIME-Version: 1.0
Content-Type: multipart/alternative;
 boundary="------------F142713BE64E3D9019D87845" 
Content-Language: en-US

This is a multi-part message in MIME format.
--------------F142713BE64E3D9019D87845
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 8bit

— — — —

--------------F142713BE64E3D9019D87845
Content-Type: text/html; charset=utf-8
Content-Transfer-Encoding: base64

PGh0bWw+DQogIDxoZWFkPg0KICAgIDxtZXRhIGh0dHAtZXF1aXY9ImNvbnRlbnQtdHlwZSIg
Y29udGVudD0idGV4dC9odG1sOyBjaGFyc2V0PXV0Zi04Ij4NCiAgPC9oZWFkPg0KICA8Ym9k
eSB0ZXh0PSIjMDAwMDAwIiBiZ2NvbG9yPSIjRkZGRkZGIj4NCiAgICA8cD48c3BhbiBzdHls
ZT0iY29sb3I6IHJnYig2OCwgNjgsIDY4KTsgZm9udC1mYW1pbHk6ICZxdW90O09wZW4gU2Fu
cyZxdW90Oywgc2Fucy1zZXJpZjsgZm9udC1zaXplOiAxMy41cHg7IGZvbnQtc3R5bGU6IG5v
cm1hbDsgZm9udC12YXJpYW50LWxpZ2F0dXJlczogbm9ybWFsOyBmb250LXZhcmlhbnQtY2Fw
czogbm9ybWFsOyBmb250LXdlaWdodDogNDAwOyBsZXR0ZXItc3BhY2luZzogbm9ybWFsOyBv
cnBoYW5zOiAyOyB0ZXh0LWFsaWduOiBzdGFydDsgdGV4dC1pbmRlbnQ6IDBweDsgdGV4dC10
cmFuc2Zvcm06IG5vbmU7IHdoaXRlLXNwYWNlOiBwcmUtd3JhcDsgd2lkb3dzOiAyOyB3b3Jk
LXNwYWNpbmc6IDBweDsgLXdlYmtpdC10ZXh0LXN0cm9rZS13aWR0aDogMHB4OyBiYWNrZ3Jv
dW5kLWNvbG9yOiByZ2JhKDY4LCA2OCwgNjgsIDAuMDgpOyB0ZXh0LWRlY29yYXRpb24tc3R5
bGU6IGluaXRpYWw7IHRleHQtZGVjb3JhdGlvbi1jb2xvcjogaW5pdGlhbDsgZGlzcGxheTog
aW5saW5lICFpbXBvcnRhbnQ7IGZsb2F0OiBub25lOyI+4oCUIDwvc3Bhbj48c3BhbiBzdHls
ZT0iY29sb3I6IHJnYig2OCwgNjgsIDY4KTsgZm9udC1mYW1pbHk6ICZxdW90O09wZW4gU2Fu
cyZxdW90Oywgc2Fucy1zZXJpZjsgZm9udC1zaXplOiAxMy41cHg7IGZvbnQtc3R5bGU6IG5v
cm1hbDsgZm9udC12YXJpYW50LWxpZ2F0dXJlczogbm9ybWFsOyBmb250LXZhcmlhbnQtY2Fw
czogbm9ybWFsOyBmb250LXdlaWdodDogNDAwOyBsZXR0ZXItc3BhY2luZzogbm9ybWFsOyBv
cnBoYW5zOiAyOyB0ZXh0LWFsaWduOiBzdGFydDsgdGV4dC1pbmRlbnQ6IDBweDsgdGV4dC10
cmFuc2Zvcm06IG5vbmU7IHdoaXRlLXNwYWNlOiBwcmUtd3JhcDsgd2lkb3dzOiAyOyB3b3Jk
LXNwYWNpbmc6IDBweDsgLXdlYmtpdC10ZXh0LXN0cm9rZS13aWR0aDogMHB4OyBiYWNrZ3Jv
dW5kLWNvbG9yOiByZ2JhKDY4LCA2OCwgNjgsIDAuMDgpOyB0ZXh0LWRlY29yYXRpb24tc3R5
bGU6IGluaXRpYWw7IHRleHQtZGVjb3JhdGlvbi1jb2xvcjogaW5pdGlhbDsgZGlzcGxheTog
aW5saW5lICFpbXBvcnRhbnQ7IGZsb2F0OiBub25lOyI+4oCUIDwvc3Bhbj48c3BhbiBzdHls
ZT0iY29sb3I6IHJnYig2OCwgNjgsIDY4KTsgZm9udC1mYW1pbHk6ICZxdW90O09wZW4gU2Fu
cyZxdW90Oywgc2Fucy1zZXJpZjsgZm9udC1zaXplOiAxMy41cHg7IGZvbnQtc3R5bGU6IG5v
cm1hbDsgZm9udC12YXJpYW50LWxpZ2F0dXJlczogbm9ybWFsOyBmb250LXZhcmlhbnQtY2Fw
czogbm9ybWFsOyBmb250LXdlaWdodDogNDAwOyBsZXR0ZXItc3BhY2luZzogbm9ybWFsOyBv
cnBoYW5zOiAyOyB0ZXh0LWFsaWduOiBzdGFydDsgdGV4dC1pbmRlbnQ6IDBweDsgdGV4dC10
cmFuc2Zvcm06IG5vbmU7IHdoaXRlLXNwYWNlOiBwcmUtd3JhcDsgd2lkb3dzOiAyOyB3b3Jk
LXNwYWNpbmc6IDBweDsgLXdlYmtpdC10ZXh0LXN0cm9rZS13aWR0aDogMHB4OyBiYWNrZ3Jv
dW5kLWNvbG9yOiByZ2JhKDY4LCA2OCwgNjgsIDAuMDgpOyB0ZXh0LWRlY29yYXRpb24tc3R5
bGU6IGluaXRpYWw7IHRleHQtZGVjb3JhdGlvbi1jb2xvcjogaW5pdGlhbDsgZGlzcGxheTog
aW5saW5lICFpbXBvcnRhbnQ7IGZsb2F0OiBub25lOyI+4oCUIDwvc3Bhbj48c3BhbiBzdHls
ZT0iY29sb3I6IHJnYig2OCwgNjgsIDY4KTsgZm9udC1mYW1pbHk6ICZxdW90O09wZW4gU2Fu
cyZxdW90Oywgc2Fucy1zZXJpZjsgZm9udC1zaXplOiAxMy41cHg7IGZvbnQtc3R5bGU6IG5v
cm1hbDsgZm9udC12YXJpYW50LWxpZ2F0dXJlczogbm9ybWFsOyBmb250LXZhcmlhbnQtY2Fw
czogbm9ybWFsOyBmb250LXdlaWdodDogNDAwOyBsZXR0ZXItc3BhY2luZzogbm9ybWFsOyBv
cnBoYW5zOiAyOyB0ZXh0LWFsaWduOiBzdGFydDsgdGV4dC1pbmRlbnQ6IDBweDsgdGV4dC10
cmFuc2Zvcm06IG5vbmU7IHdoaXRlLXNwYWNlOiBwcmUtd3JhcDsgd2lkb3dzOiAyOyB3b3Jk
LXNwYWNpbmc6IDBweDsgLXdlYmtpdC10ZXh0LXN0cm9rZS13aWR0aDogMHB4OyBiYWNrZ3Jv
dW5kLWNvbG9yOiByZ2JhKDY4LCA2OCwgNjgsIDAuMDgpOyB0ZXh0LWRlY29yYXRpb24tc3R5
bGU6IGluaXRpYWw7IHRleHQtZGVjb3JhdGlvbi1jb2xvcjogaW5pdGlhbDsgZGlzcGxheTog
aW5saW5lICFpbXBvcnRhbnQ7IGZsb2F0OiBub25lOyI+4oCUPC9zcGFuPjwvcD4NCiAgICA8
cHJlIGNsYXNzPSJtb3otc2lnbmF0dXJlIiBjb2xzPSI3MiI+LS0gDQpKZXNzaWUgUm9zZSBM
ZWUNClBhbGFudGUgVGVjaG5vbG9neSBDb29wZXJhdGl2ZQ0KPGEgY2xhc3M9Im1vei10eHQt
bGluay1mcmVldGV4dCIgaHJlZj0iaHR0cDovL3BhbGFudGV0ZWNoLmNvb3AiPmh0dHA6Ly9w
YWxhbnRldGVjaC5jb29wPC9hPg0KKDg4OCkgNDgxLTM2NTUgeC4gMTA2PC9wcmU+DQogIDwv
Ym9keT4NCjwvaHRtbD4NCg==
--------------F142713BE64E3D9019D87845--

current redmine version:

Environment:
  Redmine version                3.4.3.stable
  Ruby version                   2.2.5-p319 (2016-04-26) [x86_64-linux]
  Rails version                  4.2.8
  Environment                    production
  Database adapter               Mysql2
SCM:
  Bazaar                         2.7.0
  Git                            2.1.4
  Filesystem                     
  GitRemote                      2.1.4
Redmine plugins:
  clipboard_image_paste          1.12
  redmine_git_remote             0.0.1
  redmine_local_avatars          1.0.3
  redmine_mattermost             0.6
  redmine_stealth                0.6.0
  redmine_wiki_extensions        0.8.1

MTA has same version of ruby

Actions
Copy link
 #4

Updated by Jessie Lee almost 7 years ago

ran irb rdm-mailhandler.rb

handler = RedmineMailHandler.new
#<RedmineMailHandler:0x000000010b9c48 @issue_attributes={"project"=>"catch", "tracker"=>"support"}, @verbose=true, @url="REDACTED", @key="REDACTED">
exit(handler.submit(STDIN.read))
Posting to https://REDACTED/mail_handler...
Response received: 422
Request was denied by your Redmine server. Possible reasons: email is sent from an invalid email address or is missing some information.

Actions
Copy link
 #5

Updated by Michael Schaefer almost 7 years ago

In can confirm the issue in 3.4.3.stable. The issue didn't occur in 3.4.1. After updating to 3.4.3 I'm experiencing the same issue as mentioned by Jessie Lee.
I ran a few quick test and for the exact same E-Mail I get an

MailHandler: ignoring email from unknown user []

if

Content-Transfer-Encoding: 8bit

while

Content-Transfer-Encoding: 7bit

or

Content-Transfer-Encoding: quoted-printable

works as expected an opens a ticket for the correct user. However I seems to depend on the content itself. Not all message with the 8bit enconding get bounced.

Environment:
  Redmine version                3.4.3.stable
  Ruby version                   2.3.1-p112 (2016-04-26) [x86_64-freebsd10.3]
  Rails version                  4.2.8
  Environment                    production
  Database adapter               Mysql2
SCM:
  Git                            2.14.1
  Filesystem                     
Redmine plugins:
  redmine_checklists             3.1.10
  redmine_default_custom_query   1.3.0
  redmine_ldap_sync              2.0.9.devel

Actions
Copy link
 #6

Updated by Michael Schaefer almost 7 years ago

I have to correct myself. It seems that also for 7bit encoding I can trigger the bug. If for instance sending a mail which contains a German Umlaut äüö...

Actions
Copy link
 #7

Updated by Toshi MARUYAMA almost 7 years ago

  • Description updated (diff)
Actions
Copy link
 #8

Updated by Toshi MARUYAMA almost 7 years ago

  • Status changed from New to Needs feedback

Julien TEHERY wrote:

Migrating our production redmine from 3.2.1 to 3.2.8 , we noticed that some emails with french sepcial characters were creating empty issues.

Try downgrading mail gem.

diff --git a/Gemfile b/Gemfile
--- a/Gemfile
+++ b/Gemfile
@@ -20,7 +20,7 @@ gem "actionpack-xml_parser" 
 gem "roadie-rails", "~> 1.1.1" 
 gem "roadie", "~> 3.2.1" 
 gem "mimemagic" 
-gem "mail", "~> 2.6.6" 
+gem "mail", "= 2.6.4" 

 gem "nokogiri", (RUBY_VERSION >= "2.1" ? "~> 1.7.2" : "~> 1.6.8")
 gem "i18n", "~> 0.7.0"
Actions
Copy link
 #9

Updated by Toshi MARUYAMA almost 7 years ago

Sorry, 3.2.8 Gemfile does not have mail gem entry.
Try this.

diff --git a/Gemfile b/Gemfile
--- a/Gemfile
+++ b/Gemfile
@@ -17,6 +17,7 @@ gem "actionpack-xml_parser" 
 gem "roadie-rails", "~> 1.1.1" 
 gem "roadie", "~> 3.2.1" 
 gem "mimemagic" 
+gem "mail", "= 2.6.4" 

 gem "nokogiri", (RUBY_VERSION >= "2.1" ? "~> 1.7.2" : "~> 1.6.8")
 gem "i18n", "~> 0.7.0"
Actions
Copy link
 #10

Updated by Jessie Lee almost 7 years ago

can confirm that downgrading to 2.6.4 mail gem worked for me.

cheers

Actions
Copy link
 #11

Updated by Toshi MARUYAMA almost 7 years ago

  • Subject changed from Issue created from mail_handler with description empty from to Issue created from mail_handler with description empty from
Actions
Copy link
 #12

Updated by Toshi MARUYAMA almost 7 years ago

I think you would better report to mail gem team.
https://github.com/mikel/mail/issues

Actions
Copy link
 #13

Updated by Michael Schaefer almost 7 years ago

binding mail gem to 2.6.4 (was 2.7.0. in my rvm env) solved it for 3.4.3 as well.

Actions
Copy link
 #14

Updated by Go MAEDA almost 7 years ago

  • Priority changed from High to Normal
  • Target version set to 3.2.9
Actions
Copy link
 #15

Updated by Toshi MARUYAMA almost 7 years ago

  • Target version deleted (3.2.9)

I have removed target version because it seems mail gem 2.6.5 and 2.6.6 have Ruby 2.4.0 compatibility and security fix.
https://github.com/mikel/mail/blob/2.6.6/CHANGELOG.rdoc#version-266---2017-06-09-jeremy-daer-jeremydaergmailcom

Actions
Copy link
 #16

Updated by Toshi MARUYAMA almost 7 years ago

Julien TEHERY wrote:

Here are some precisions about the incoming emails:

Looking at the source code of the emails, here is what i noticed.

  • BAD INCOMING MAIL TURNED INTO EMPTY ISSUE
    Content-Type: text/plain; charset=utf-8; format=flowed
    Content-Transfer-Encoding: 8bit
  • GOOD MAIL TURNED INTO FULLFILLED ISSUE
    Content-Type: text/plain; charset=UTF-8
    Content-Transfer-Encoding: quoted-printable

I really don't have any idea why 8bit encoding would cause such a behavior, but the fact is that all
bad emails are formated this way.

Could you give us or mail gem team good and bad mail examples?
Actions
Copy link
 #17

Updated by Go MAEDA over 6 years ago

  • Related to Defect #29365: MailHandlerTest#test_add_issue_with_localized_attributes fails with mail gem 2.7.0 added
Actions
Copy link
 #18

Updated by Go MAEDA about 6 years ago

  • Target version set to Candidate for next minor release
Actions
Copy link
 #19

Updated by Go MAEDA about 6 years ago

Actions
Copy link
 #20

Updated by Go MAEDA about 6 years ago

  • Status changed from Needs feedback to Confirmed

The team in my company investigated this issue and found that the problem occurs when the following conditions are met:

  • mail 2.7.0 is installed (no problem with 2.6.x or 2.7.1)
  • Line terminators of the message are LF (according to RFC 5322, those must be CRLF)
  • Encoding of the body is UTF-8 and the body contains Unicode characters

They will soon post a test that illustrates the problem.

Actions
Copy link
 #21

Updated by Yuichi HARADA about 6 years ago

Go MAEDA wrote:

  • mail 2.7.0 is installed (no problem with 2.6.x or 2.7.1)
  • Line terminators of the message are LF (according to RFC 5322, those must be CRLF)
  • Encoding of the body is UTF-8 and the body contains Unicode characters

I created a test code.

  • This is a test code for detecting a problem.
  • mail 2.7.0 fails, however mail 2.7.1 succeeds.
  • With this test code, you can confirm that this problem is solved by updating to mail 2.7.1.
Actions
Copy link
 #22

Updated by Go MAEDA about 6 years ago

  • Status changed from Confirmed to Closed
  • Target version deleted (Candidate for next minor release)
  • Resolution set to Fixed

As I wrote in #27691#note-20, the cause of this problem is mail gem 2.7.0.

With the test written by Yuichi HARADA, we can confirm that the problem is peculiar to mail gem 2.7.0. We have updated the mail gem to 2.7.1 (#29443) and the problem will never occur.

mail gem 2.7.1 (PASS):

$ ruby test/unit/mail_handler_test.rb -n test_add_issue_with_utf8_body_and_lf_line_terminator
Run options: -n test_add_issue_with_utf8_body_and_lf_line_terminator --seed 34645

# Running:

.

Finished in 3.034103s, 0.3296 runs/s, 0.9888 assertions/s.
1 runs, 3 assertions, 0 failures, 0 errors, 0 skips

mail gem 2.7.0 (FAIL):

$ ruby test/unit/mail_handler_test.rb -n test_add_issue_with_utf8_body_and_lf_line_terminator
Run options: -n test_add_issue_with_utf8_body_and_lf_line_terminator --seed 6489

# Running:

F

Failure:
MailHandlerTest#test_add_issue_with_utf8_body_and_lf_line_terminator [test/unit/mail_handler_test.rb:1175]:
Expected false to be truthy.

bin/rails test test/unit/mail_handler_test.rb:1167

Finished in 1.992780s, 0.5018 runs/s, 0.5018 assertions/s.
1 runs, 1 assertions, 1 failures, 0 errors, 0 skips

mail gem 2.6.6 (PASS):

$ ruby test/unit/mail_handler_test.rb -n test_add_issue_with_utf8_body_and_lf_line_terminator
Run options: -n test_add_issue_with_utf8_body_and_lf_line_terminator --seed 7210

# Running:

.

Finished in 2.368979s, 0.4221 runs/s, 1.2664 assertions/s.
1 runs, 3 assertions, 0 failures, 0 errors, 0 skips

Actions
Copy link
 #23

Updated by Go MAEDA about 6 years ago

Actions
Copy link
 #24

Updated by Go MAEDA about 6 years ago

Actions
Copy link

Also available in: Atom PDF