Redmine

If you're a fool then so am I, I am using "STOP" and "====" for my delimiters and tested STOP with no dice..

I think the issue for me was that strip_tags was leaving whitespace in front of the delimiter, so I changed app/models/mail_handler.rb to allow whitespace before the delimiter:

regex = Regexp.new("^\s*(#{ delimiters.join('|') })\s*[\r\n].*", Regexp::MULTILINE)

Seems to have fixed it.

sigh Danm, I knew the day would come when I would really have to learn regular expressions :) http://xkcd.com/208/

I will try that, many thanks for the hints guys!

Hm... Didn't work for me. Anyone has another idea?

Another thing that I wonder: all line breaks are stripped from the mail, is that normal behavior??

Well that would explain why the regex isn't working, if the line breaks are stripped.

Jan Wedekind wrote:

Another thing that I wonder: all line breaks are stripped from the mail, is that normal behavior??

The line breaks might get eaten by the html rendering once you look at it in the browser, but redmine shouldn't strip them. I'm not sure if the Regex isn't applied after the html has been stripped, have you tried putting just REPLY ABOVE THIS LINE in your line match thingie?

Felix Schäfer wrote:

The line breaks might get eaten by the html rendering once you look at it in the browser, but redmine shouldn't strip them. I'm not sure if the Regex isn't applied after the html has been stripped, have you tried putting just REPLY ABOVE THIS LINE in your line match thingie?

Yes, I did that (w/o any special characters in the line). I even put that in the email by hand.

I will put back the mail_handler from the trunk to check if I have touched something without knowing that screws that up.

Is there any prior bad experience with mails sent from Apple Mail or pulled from Exchange 2007 via IMAP? Exchange tends to do weird things (just learned for example, that it strips the pictures from the notification mail grrr!)

Jan Wedekind wrote:

Is there any prior bad experience with mails sent from Apple Mail

Works well here (in general apple will either adhere to a standard or break it enough that they call it something else).

Felix Schäfer wrote:

Works well here (in general apple will either adhere to a standard or break it enough that they call it something else).

Well put :).

Still, it won't work here. Attached a screen showing one reply sent from my Gmail account and one from Apple Mail hooked up to Exchange 2007. As you can see, the Gmail one seems almost fine, but I note that it also stripped the first line break (the one before "On Oct...") and the line break in the footer of the notification ("... in it.Change...").

The one from Apple Mail is completely stripped of all line breaks.

Before, I put back the mail_handler.rb from the trunk (although I hadn't touched it afaik). I almost removed any string from the "Truncate after" field in the settings. Then I restarted the server.

Stupid question: according to the Wiki, the "Enable WS for incoming emails" is only relevant for direct mail forwards, right? In any case, even with that activated it doesn't work.

Any ideas where I can start digging?? Since it doesn't appear to be a general bug, I surely don't want to file a bug report.

Many thanks,
Jan

Ok, this is getting a bit esoteric.

• Replies in plain text are parsed and handled correctly, no matter from where I send them
• Replies from Outlook 2007 sent via the same Exchange Server I use with Apple Mail are handled correctly

I have no idea what gets garbled in the coding of the email sent from Apple Mail via Exchange.

Felix, could you point me to where this body is cut? Is is l266ff in mail_handler.rb?

Are there even any line breaks to begin with? e.g, if the html looks like this

<p>This is a test.</p><p>REPLY ABOVE THIS LINE</p>

Then there simply are no line breaks. In which case I'd add some code before the strip_tags to add line breaks.

Colin Mollenhour wrote:

Are there even any line breaks to begin with? e.g, if the html looks like this
[...] Then there simply are no line breaks. In which case I'd add some code before the strip_tags to add line breaks.

Hm... Not sure if I fully understand. Shouldn't it also take paragraph breaks into account and not only <br>? Did you mean adding a <br>? If so, where exactly? Sorry for being a bit slow and stupid :(

Felix, I can narrow down the problem to Apple Mail. Gmail and Outlook work nicely. Apple Mail does only work, if the reply is sent in plain text. Can you confirm that it works using Rich Text Mails from Apple Mail?

Look at the raw HTML generated by Apple Mail, I suspect it is lacking whitespace and I suppose strip_tags is not smart enough to add newlines.

The raw HTML looks positively ghastly (see below). So, apparently, Redmine strips the whole HTML code in there including the <p> tags.

Stupid question: why is Redmine not simply interpreting the "Content-Type: text/plain;" part of the email message? Wouldn't that be a hell lot easier? Or can I hardcode that behavior somewhere?

Many thanks!

Content-Type: text/html; charset="us-ascii" 
Content-Transfer-Encoding: quoted-printable

<html><head><base href=3D"x-msg://526/"></head><body style=3D"word-wrap: br=
eak-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; =
"><div>Body from the mail sent from Apple Mail.</div><br><div><div>On Oct 1=
8, 2010, at 18:19, IRIS Project Management wrote:</div><br class=3D"Apple-i=
nterchange-newline"><blockquote type=3D"cite"><span class=3D"Apple-style-sp=
an" style=3D"border-collapse: separate; font-family: Helvetica; font-style:=
 normal; font-variant: normal; font-weight: normal; letter-spacing: normal;=
 line-height: normal; orphans: 2; text-indent: 0px; text-transform: none; w=
hite-space: normal; widows: 2; word-spacing: 0px; -webkit-border-horizontal=
-spacing: 0px; -webkit-border-vertical-spacing: 0px; -webkit-text-decoratio=
ns-in-effect: none; -webkit-text-size-adjust: auto; -webkit-text-stroke-wid=
th: 0px; font-size: medium; "><div style=3D"font-family: Verdana, sans-seri=
f; font-size: 0.8em; color: rgb(72, 72, 72); "><p style=3D"font-size: 10px;=
 color: grey; ">+++</p><br>Issue #331 has been reported by Jan Wedekind.<hr=
 style=3D"width: 1168px; height: 1px; background-image: initial; background=
-attachment: initial; background-origin: initial; background-clip: initial;=
 background-color: rgb(204, 204, 204); border-top-width: 0px; border-right-=
width: 0px; border-bottom-width: 0px; border-left-width: 0px; border-style:=
 initial; border-color: initial; background-position: initial initial; back=
ground-repeat: initial initial; "><h1 style=3D"font-family: 'Trebuchet MS',=
 Verdana, sans-serif; margin-top: 0px; margin-right: 0px; margin-bottom: 0p=
x; margin-left: 0px; font-size: 1.2em; "><a href=3D"http://redmine.iris.cat=
/issues/331" style=3D"color: rgb(42, 86, 133); ">Task #331: Test</a></h1><u=
l><li>Author: Jan Wedekind</li><li>Status: New</li><li>Priority: Normal</li=
><li>Assigned to: Jan Wedekind</li><li>Due date:</li><li>Target milestone:<=
/li></ul><p>Here is the description.</p><hr style=3D"width: 1168px; height:=
 1px; background-image: initial; background-attachment: initial; background=
-origin: initial; background-clip: initial; background-color: rgb(204, 204,=
 204); border-top-width: 0px; border-right-width: 0px; border-bottom-width:=
 0px; border-left-width: 0px; border-style: initial; border-color: initial;=
 background-position: initial initial; background-repeat: initial initial; =
"><span class=3D"footer" style=3D"font-size: 0.8em; font-style: italic; "><=
p>You have received this notification because you have either subscribed to=
 it, or are involved in it.<br>Change your notification preferences:<span c=
lass=3D"Apple-converted-space">&nbsp;</span><a class=3D"external" href=3D"h=
ttp://redmine.iris.cat/my/account" style=3D"color: rgb(42, 86, 133); ">http=
://redmine.iris.cat/my/account</a>.<br>Problems or questions:<span class=3D=
"Apple-converted-space">&nbsp;</span><a class=3D"email" href=3D"mailto:redm=
ine@iris.cat" style=3D"color: rgb(42, 86, 133); ">redmine@iris.cat</a>.</p>=
</span></div></span></blockquote></div><br></body></html>=

Is there a plain text part attached? If there is Redmine should already be using it.

Jan Wedekind wrote:

Felix, I can narrow down the problem to Apple Mail. Gmail and Outlook work nicely. Apple Mail does only work, if the reply is sent in plain text. Can you confirm that it works using Rich Text Mails from Apple Mail?

I only use plain text, can't say anything about the rich text ones.

Thanks Felix, thought so!

Colin, yes, there is a MIME part that is plain text. Here the full raw text of the mail that of which posted just the HTML abvove. The "+++" is where it should cut.

I was wondering why Redmine wasn't just using that part anyway. Maybe this is a bug after all?

From: Jan Wedekind <jwedekind@company.com>
To: Redmine <redmine@company.com>
Content-Class: urn:content-classes:message
Date: Mon, 18 Oct 2010 12:19:01 -0400
Subject: Re: [Test Project - Task #331] (New) Test
Thread-Topic: [Test Project - Task #331] (New) Test
Thread-Index: Actu4C1ghl4XJ46BS/2jIhw4rMr8dw==
Message-ID: <CEFFDD21-3054-46CF-B8B4-348D111BB345@company.com>
References: <redmine.issue-331.20101018181919@company.com>
In-Reply-To: <redmine.issue-331.20101018181919@company.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-Exchange-Organization-SCL: -1
X-MS-TNEF-Correlator:
Content-Type: multipart/alternative;
    boundary="_000_CEFFDD21305446CFB8B4348D111BB345iriscat_" 
MIME-Version: 1.0

--_000_CEFFDD21305446CFB8B4348D111BB345companycom_
Content-Type: text/plain; charset="us-ascii" 
Content-Transfer-Encoding: quoted-printable

Body from the mail sent from Apple Mail.

On Oct 18, 2010, at 18:19, IRIS Project Management wrote:

+++

Issue #331 has been reported by Jan Wedekind.
________________________________
Task #331: Test<http://redmine.company.com/issues/331>

 *   Author: Jan Wedekind
 *   Status: New
 *   Priority: Normal
 *   Assigned to: Jan Wedekind
 *   Due date:
 *   Target milestone:

Here is the description.

________________________________

You have received this notification because you have either subscribed to i=
t, or are involved in it.
Change your notification preferences: http://redmine.company.com/my/account.
Problems or questions: redmine@company.com<mailto:redmine@company.com>.

--_000_CEFFDD21305446CFB8B4348D111BB345companycom_
Content-Type: text/html; charset="us-ascii" 
Content-Transfer-Encoding: quoted-printable

<html><head><base href=3D"x-msg://526/"></head><body style=3D"word-wrap: br= (...rest of the HTML part)

--_000_CEFFDD21305446CFB8B4348D111BB345iriscat_--

It's no supper biggie because I mainly activated it now as a fallback for external partners who don't use Redmine daily and might just hit "Reply" on the notification. So the main issue really is getting the info into the system and Redmine adds it as a comment to the right issue (and even the attachment!). So I am already happy.

So now my question is more if it's worth filing a bug report on that.

Felix, maybe you can make a quick check with Apple Mail using Rich Text? Just to see if that is a common problem our just ours.

The code is supposed to prefer the text/plain part, so I definitely think you should file a bug.

    parts = @email.parts.collect {|c| (c.respond_to?(:parts) && !c.parts.empty?) ? c.parts : c}.flatten
    if parts.empty?
      parts << @email
    end
    plain_text_part = parts.detect {|p| p.content_type == 'text/plain'}

I filed a bug report: #6705. Thanks again already for all your input and help, greatly appreciated.