https://www.redmine.org/https://www.redmine.org/favicon.ico?16793021292014-05-23T11:49:08ZRedmineRedmine - Feature #16962: Better handle html-only emailshttps://www.redmine.org/issues/16962?journal_id=564422014-05-23T11:49:08ZFelix Schäfer
<ul><li><strong>File</strong> <a href="/attachments/11656">16962.patch</a> <a class="icon-only icon-download" title="Download" href="/attachments/download/11656/16962.patch">16962.patch</a> added</li></ul><p>The attached patch expands the html-only email used for testing by markup similar to what we have experienced at <a href="https://plan.io/redmine" class="external">Planio</a>. A growing number of users send html-only emails, especially Outlook users.</p>
<p>The test run with the current MailHandler results in:</p>
<pre>
p {font-size:12.0pt;}\r\n\r\n\r\nThis is a html-only email.With a titleand a paragraph.
</pre>
<p>There is content from the <code>head</code> and the actual email text is all squashed together.</p>
<p>The result with the patch applied is:</p>
<pre>
This is a html-only email.\r\nWith a title\r\n\r\nand a paragraph.
</pre>
<p>The <a href="https://github.com/flavorjones/loofah" class="external">loofah</a> gem parses the html based on Nokogiri and tries to handle tags a little more intelligently than just removing them.</p> Redmine - Feature #16962: Better handle html-only emailshttps://www.redmine.org/issues/16962?journal_id=564432014-05-23T11:57:22ZJan from Planio www.plan.io
<ul><li><strong>Tracker</strong> changed from <i>Feature</i> to <i>Patch</i></li><li><strong>Status</strong> changed from <i>New</i> to <i>Needs feedback</i></li><li><strong>Target version</strong> set to <i>Candidate for next minor release</i></li></ul> Redmine - Feature #16962: Better handle html-only emailshttps://www.redmine.org/issues/16962?journal_id=564442014-05-23T12:07:37ZFelix Schäfer
<ul><li><strong>File</strong> <a href="/attachments/11657">16962.2.patch</a> <a class="icon-only icon-download" title="Download" href="/attachments/download/11657/16962.2.patch">16962.2.patch</a> added</li></ul><p>Here is an updated patch to only convert html parts to text and leave text parts alone.</p> Redmine - Feature #16962: Better handle html-only emailshttps://www.redmine.org/issues/16962?journal_id=564452014-05-23T12:16:00ZToshi MARUYAMA
<ul><li><strong>Tracker</strong> changed from <i>Patch</i> to <i>Feature</i></li></ul> Redmine - Feature #16962: Better handle html-only emailshttps://www.redmine.org/issues/16962?journal_id=564482014-05-23T12:17:35ZToshi MARUYAMA
<ul><li><strong>Has duplicate</strong> <i><a class="issue tracker-2 status-5 priority-4 priority-default closed" href="/issues/16034">Feature #16034</a>: Support for HTML in incoming mails</i> added</li></ul> Redmine - Feature #16962: Better handle html-only emailshttps://www.redmine.org/issues/16962?journal_id=567882014-06-13T08:49:17ZJoe Darkless
<ul></ul><p>I think you are mixing things up. So to shed some light on this problem I'll be verbose. (if there are any misstatements feel free to correct them.) (Oh and I have redmine version 2.4.0)</p>
<p>There are 2 issues concerning HTML formatting from incomming mails.</p>
<p><strong>The first one:</strong></p>
<a name="Parsing-incoming-mails-and-storing-them-into-database"></a>
<h2 >Parsing incoming mails and storing them into database<a href="#Parsing-incoming-mails-and-storing-them-into-database" class="wiki-anchor">¶</a></h2>
<p>This process is made in app/models/mail_handler.rb. Around line 400 there is a method called plain_text_body, which (imho) takes care of formatting/stripping email body and is called by receive_imap method (indirectly)</p>
<p>This method really sux and doesn't work as it should at all. There can be 3 scenario email. Plaintext only email, HTML only email, plaintext with html both included in email.</p>
<p>If the plaintext email is sent, this method does its job more or less. Same with both formats included. It just grabs plaintext and its done. And It really had problem with HTML only email. And <strong>thats</strong> what you stated patch here is for... (though thew me an error that some method of loofa doesn't exist - but I think thats due to my version of redmine) Without the patch the HTML mail is just stored in DB as is, without any striping. Anyway it shoud do the parsing for new lines (or BRs) better.</p>
<p><strong>The second one</strong></p>
<a name="Displaying-description-content-on-issue-page"></a>
<h2 >Displaying description content on issue page<a href="#Displaying-description-content-on-issue-page" class="wiki-anchor">¶</a></h2>
<p>This bug is relevant mainly to CKEditor. If you have disabled CKEditor for Project the "plain-text" mail created issues are displayed corectly. But if you turn the "WYSIWYG" editor on there are no LINE BREAKS and the text is rendered on one line. Referring to issue 12025 (<a class="external" href="http://www.redmine.org/issues/12025">http://www.redmine.org/issues/12025</a>) there is probbably no processing of plain_text to HTML to display the content properly. And this bug is still not solved by stated patch.</p> Redmine - Feature #16962: Better handle html-only emailshttps://www.redmine.org/issues/16962?journal_id=567892014-06-13T09:08:40ZFelix Schäfer
<ul></ul><p>Joe, you are right that this patch only treats your stated point 1, I don't think I stated otherwise either.</p>
<p>Your second point is irrelevant to core Redmine currently, as (at least last time I checked) Redmine ever only stores text with markup (textile or markdown) but at no time full html. If you use a rich text editor with your Redmine or if your Redmine stores plain HTML, it comes from a plugin and that would be the concern of the plugin author, not of core Redmine. This second point is thus not relevant here and I will not engage in further discussion about it in this ticket. If you think this is relevant to Redmine, please open another ticket about it.</p> Redmine - Feature #16962: Better handle html-only emailshttps://www.redmine.org/issues/16962?journal_id=567902014-06-13T09:26:05ZFelix Schäfer
<ul></ul><p>Joe Darkless wrote:</p>
<blockquote>
<a name="Parsing-incoming-mails-and-storing-them-into-database"></a>
<h2 >Parsing incoming mails and storing them into database<a href="#Parsing-incoming-mails-and-storing-them-into-database" class="wiki-anchor">¶</a></h2>
<p>This process is made in app/models/mail_handler.rb. Around line 400 there is a method called plain_text_body, which (imho) takes care of formatting/stripping email body and is called by receive_imap method (indirectly)</p>
</blockquote>
<p><code>MailHandler#plain_text_body</code> is exactly the method the proposed patch changes.</p>
<blockquote>
<p>This method really sux</p>
</blockquote>
<p>That's a little more harsh than what I'd say, but yes.</p>
<blockquote>
<p>and doesn't work as it should at all.</p>
</blockquote>
<p>No, it works as it should: provide a plain-text version of an incoming email's body. You might not agree with the method to achieve that goal, but it does achieve it.</p>
<blockquote>
<p>There can be 3 scenario email. Plaintext only email, HTML only email, plaintext with html both included in email.</p>
<p>If the plaintext email is sent, this method does its job more or less. Same with both formats included. It just grabs plaintext and its done. And It really had problem with HTML only email.</p>
</blockquote>
<p>If the email has at least one text-only part, then all text parts are kept. If the email has no text part but html parts, then all html parts are kept. Any html part then is stripped of all tags before being returned <a class="source" href="https://www.redmine.org/projects/redmine/repository/svn/entry/app/models/mail_handler.rb#L431">source:app/models/mail_handler.rb#L431</a>, but this tag-stripping is very basic and lacking.</p>
<blockquote>
<p>And <strong>thats</strong> what you stated patch here is for...</p>
</blockquote>
<p>Yes. The patch changes the handling of said html parts and instead of just blindly removing tags and tries to make some more sense of the html markup (still at a very basic level though).</p>
<p>I don't know what in the title or description made you think otherwise.</p>
<blockquote>
<p>(though thew me an error that some method of loofa doesn't exist - but I think thats due to my version of redmine)</p>
</blockquote>
<p>It shouldn't. Did you run <code>bundle install</code> after applying the patch and restarted your Redmine instance before trying it out? If this doesn't help, could you please provide me with the error you got?</p>
<blockquote>
<p>Without the patch the HTML mail is just stored in DB as is, without any striping.</p>
</blockquote>
<p>No. Vanilla Redmine doesn't store plain HTML anywhere in the DB, if that is the case for you this is caused by a plugin.</p>
<blockquote>
<p>Anyway it shoud do the parsing for new lines (or BRs) better.</p>
</blockquote>
<p>That is exactly what this patch addresses: currently the <code>MailHandler</code> will just drop for example <code><br/></code>, which means this</p>
<pre><code class="html syntaxhl">This is the first line.<span class="nt"><br/></span>This is the second line.</code></pre>
<p>will currently just become</p>
<pre>This is the first line.This is the second line.</pre>
<p>whereas with the patch it would become</p>
<pre>This is the first line.
This is the second line.</pre>
<p>I hope I was able to clear up any misunderstanding you had about this ticket or the attached patch.</p> Redmine - Feature #16962: Better handle html-only emailshttps://www.redmine.org/issues/16962?journal_id=567912014-06-13T10:24:25ZJoe Darkless
<ul></ul><p>Hello,<br />thanks for the fast response. I admit that I was a bit harsh, because as someone who learnt ruby during last 3 days by just diggin in the code I was upset. So sorry for that, I didn't mean it.</p>
<p>The reason that I have included reference to the second problem (which is connected to external plugin) was because to fix problem you need to fix both of the issues, and I think that also other shoud know that there is a problem. And I definitely understad that this has nothing to do with redmine core. So as a quickfix for the second stated issue I'm going to redirect others here: <a class="external" href="https://github.com/a-ono/redmine_ckeditor/issues/89">https://github.com/a-ono/redmine_ckeditor/issues/89</a></p>
<p>Ok, and concerning the first issue, which is related to redmine core:<br />Yes I have run bundle install respectively (rvm 2.0.0 exec bundle install) And it finished sucessfuly with loofa gem installed.<br />Yes I have restarted httpd server (with passenger)<br />I have posted the error on pastebin <a class="external" href="http://pastebin.com/Cp3fCMhM">http://pastebin.com/Cp3fCMhM</a> (not sure about pasting politic so please feel free to post it here or include as a file)<br />I really wasn't able to troubleshoot this error with my 3day knowledge of ruby and didn't find relevant issue on google.</p> Redmine - Feature #16962: Better handle html-only emailshttps://www.redmine.org/issues/16962?journal_id=567922014-06-13T10:39:23ZFelix Schäfer
<ul></ul><p>Joe Darkless wrote:</p>
<blockquote>
<p>thanks for the fast response. I admit that I was a bit harsh, because as someone who learnt ruby during last 3 days by just diggin in the code I was upset. So sorry for that, I didn't mean it.</p>
</blockquote>
<p>The Redmine code has a lot of baggage and really shows its age in some parts, so it's probably not the best codebase to get started with Ruby and Rails (sadly). I'd say kudos for keeping at it for 3 days :-)</p>
<blockquote>
<p>Ok, and concerning the first issue, which is related to redmine core:<br />Yes I have run bundle install respectively (rvm 2.0.0 exec bundle install) And it finished sucessfuly with loofa gem installed.<br />Yes I have restarted httpd server (with passenger)<br />I have posted the error on pastebin <a class="external" href="http://pastebin.com/Cp3fCMhM">http://pastebin.com/Cp3fCMhM</a> (not sure about pasting politic so please feel free to post it here or include as a file)<br />I really wasn't able to troubleshoot this error with my 3day knowledge of ruby and didn't find relevant issue on google.</p>
</blockquote>
<p>It seems I had developed the patch with Loofah 1.x and that since Loofah 2.0 has been released, which requires a little more legwork. Could you try adding</p>
<pre><code class="ruby syntaxhl"><span class="nb">require</span> <span class="s2">"loofah/helpers"</span></code></pre>
<p>at the top of the patched <code>mail_handler.rb</code>? I don't have the time to test it right now, but that should do it.</p> Redmine - Feature #16962: Better handle html-only emailshttps://www.redmine.org/issues/16962?journal_id=567932014-06-13T11:20:07ZJoe Darkless
<ul></ul><p>Felix Schäfer wrote:</p>
<blockquote>
<p>It seems I had developed the patch with Loofah 1.x and that since Loofah 2.0 has been released, which requires a little more legwork. Could you try adding</p>
<p>[...]</p>
<p>at the top of the patched <code>mail_handler.rb</code>? I don't have the time to test it right now, but that should do it.</p>
</blockquote>
<p>Yes that fixed the problem. Thank you very much.</p> Redmine - Feature #16962: Better handle html-only emailshttps://www.redmine.org/issues/16962?journal_id=568222014-06-16T06:59:50ZJoe Darkless
<ul></ul><p>Excuse me for inconvenience but I don't know how to link issue 'Related to' another record.</p>
<p>This Issue is somehow related to Feature <a class="external" href="http://www.redmine.org/issues/8462">http://www.redmine.org/issues/8462</a> and in my opinion it is good to know.</p> Redmine - Feature #16962: Better handle html-only emailshttps://www.redmine.org/issues/16962?journal_id=574972014-07-13T01:08:02ZToshi MARUYAMA
<ul><li><strong>Related to</strong> <i><a class="issue tracker-2 status-1 priority-4 priority-default" href="/issues/8462">Feature #8462</a>: Save received email as an attachment, not as a comment</i> added</li></ul> Redmine - Feature #16962: Better handle html-only emailshttps://www.redmine.org/issues/16962?journal_id=577382014-07-25T02:34:03ZYehuda Katz
<ul></ul><p>Patch 2 works for me (although I had to apply it manually, it did not patch cleanly). I also had to add the <code>require "loofah/helpers"</code> line.</p> Redmine - Feature #16962: Better handle html-only emailshttps://www.redmine.org/issues/16962?journal_id=579722014-08-10T00:11:31ZToshi MARUYAMA
<ul><li><strong>Related to</strong> <i><a class="issue tracker-1 status-5 priority-4 priority-default closed" href="/issues/12025">Defect #12025</a>: rake receive_pop3 is stripping formatting and new lines</i> added</li></ul> Redmine - Feature #16962: Better handle html-only emailshttps://www.redmine.org/issues/16962?journal_id=600782014-11-29T10:35:41ZFelix Schäfer
<ul></ul><p>Just a quick note that Rails starting with 4.2.0 will use Loofah internally <a class="external" href="http://edgeguides.rubyonrails.org/4_2_release_notes.html#html-sanitizer">http://edgeguides.rubyonrails.org/4_2_release_notes.html#html-sanitizer</a> So this will only be an additional dependency until then.</p> Redmine - Feature #16962: Better handle html-only emailshttps://www.redmine.org/issues/16962?journal_id=604422014-12-18T03:31:10ZToshi MARUYAMA
<ul><li><strong>Related to</strong> <i><a class="issue tracker-1 status-5 priority-4 priority-default closed" href="/issues/18374">Defect #18374</a>: Incoming emails are truncating CRLF</i> added</li></ul> Redmine - Feature #16962: Better handle html-only emailshttps://www.redmine.org/issues/16962?journal_id=605192014-12-23T02:53:22ZKrishna Gollamudi
<ul></ul><p>Felix Schäfer wrote:</p>
<blockquote>
<p>Just a quick note that Rails starting with 4.2.0 will use Loofah internally <a class="external" href="http://edgeguides.rubyonrails.org/4_2_release_notes.html#html-sanitizer">http://edgeguides.rubyonrails.org/4_2_release_notes.html#html-sanitizer</a> So this will only be an additional dependency until then.</p>
</blockquote>
<p>After applying this patch and reverting the changes, I am getting Gem not found errorrs. Can you please help me? Here below is the complete log.<br />.0.21/helper-scripts/passenger-spawn-server 102</p> Redmine - Feature #16962: Better handle html-only emailshttps://www.redmine.org/issues/16962?journal_id=605232014-12-23T06:53:50ZFelix Schäfer
<ul></ul><p>Make backups before applying patches.</p>
<p>Other than that, I think that running <code>bundle install</code> in your Redmine directory and restarting apache should do the trick.</p> Redmine - Feature #16962: Better handle html-only emailshttps://www.redmine.org/issues/16962?journal_id=605272014-12-23T07:42:40ZKrishna Gollamudi
<ul></ul><p>Felix Schäfer wrote:</p>
<blockquote>
<p>Make backups before applying patches.</p>
<p>Other than that, I think that running <code>bundle install</code> in your Redmine directory and restarting apache should do the trick.</p>
</blockquote>
<p>I did, but still having the issues. The Gem is installed but somehow it is not being detected. Here is my bundle env. Do you find anything missing?</p> Redmine - Feature #16962: Better handle html-only emailshttps://www.redmine.org/issues/16962?journal_id=605282014-12-23T07:46:38ZFelix Schäfer
<ul></ul><p>Check if you can run the built-in rails console or server for Redmine. If it works, the problem is with Passenger or your Passenger configuration.</p> Redmine - Feature #16962: Better handle html-only emailshttps://www.redmine.org/issues/16962?journal_id=605292014-12-23T07:47:39ZFelix Schäfer
<ul></ul><p>(and this is the wrong place to get help with your installation, as it has nothing to do with the original ticket. For further help I suggest heading to the forums.)</p> Redmine - Feature #16962: Better handle html-only emailshttps://www.redmine.org/issues/16962?journal_id=606052014-12-27T06:47:41ZKrishna Gollamudi
<ul></ul><p>Felix:</p>
<p>I am able to identify the root cause and fix it. Has to do with the ruby and gems path but my emails are still loosing the HTML tags while loading into Redmine. Does your above patch works for Redmine 2.5.2 ?</p> Redmine - Feature #16962: Better handle html-only emailshttps://www.redmine.org/issues/16962?journal_id=606082014-12-27T11:38:32ZJean-Philippe Langjp_lang@yahoo.fr
<ul><li><strong>Target version</strong> changed from <i>Candidate for next minor release</i> to <i>Candidate for next major release</i></li></ul><p>With the patch applied, new lines are inserted as expected but this doesn't solve 2. of #13209: new lines in incoming html should be removed, eg.</p>
<pre>
<p>This is
a paragraph</p>
</pre>
<p>should be converted to a single line: "This is a paragraph".</p> Redmine - Feature #16962: Better handle html-only emailshttps://www.redmine.org/issues/16962?journal_id=606092014-12-27T11:42:52ZJean-Philippe Langjp_lang@yahoo.fr
<ul></ul><p>A simple fix could be <code>gsub(/\s+/, ' ')</code> the html before processing it with Loofah.</p> Redmine - Feature #16962: Better handle html-only emailshttps://www.redmine.org/issues/16962?journal_id=606102014-12-27T14:42:48ZKrishna Gollamudi
<ul></ul><p>Jean-Philippe Lang wrote:</p>
<blockquote>
<p>With the patch applied, new lines are inserted as expected but this doesn't solve 2. of #13209: new lines in incoming html should be removed, eg.</p>
<p>[...]</p>
<p>should be converted to a single line: "This is a paragraph".</p>
</blockquote>
<p>Jean, Are you able to retain tables coming from the incoming emails?</p> Redmine - Feature #16962: Better handle html-only emailshttps://www.redmine.org/issues/16962?journal_id=606132014-12-27T14:59:12ZFelix Schäfer
<ul></ul><p>Krishna Gollamudi wrote:</p>
<blockquote>
<p>I am able to identify the root cause and fix it. Has to do with the ruby and gems path but my emails are still loosing the HTML tags while loading into Redmine. Does your above patch works for Redmine 2.5.2 ?</p>
</blockquote>
<p>That's the point: before the patch, if the incoming email has no text-part, the html-part is taken, but with all HTML tags removed. With the patch, in the case of no text-part, the html-part is taken, still with all HTML tags removed, but with certain tags replaced with newlines to keep paragraphs and alike.</p> Redmine - Feature #16962: Better handle html-only emailshttps://www.redmine.org/issues/16962?journal_id=606142014-12-27T15:23:40ZFelix Schäfer
<ul></ul><p>Jean-Philippe Lang wrote:</p>
<blockquote>
<p>With the patch applied, new lines are inserted as expected but this doesn't solve 2. of #13209: new lines in incoming html should be removed, eg.</p>
<p>[...]</p>
<p>should be converted to a single line: "This is a paragraph".</p>
</blockquote>
<p>I had not considered this case. I'll see if there is an option in loofah so that a wholesale replacing of all whitespaces in the whole document is not needed.</p> Redmine - Feature #16962: Better handle html-only emailshttps://www.redmine.org/issues/16962?journal_id=606152014-12-27T15:25:25ZFelix Schäfer
<ul></ul><p>Krishna Gollamudi wrote:</p>
<blockquote>
<p>Jean, Are you able to retain tables coming from the incoming emails?</p>
</blockquote>
<p>If you mean html tables: no, this method transforms the html-part of a text-part-less email to a somewhat equivalent text, but does not know about textile or markdown markup that might be used to render that text.</p> Redmine - Feature #16962: Better handle html-only emailshttps://www.redmine.org/issues/16962?journal_id=606172014-12-27T17:51:16ZKrishna Gollamudi
<ul><li><strong>File</strong> <a href="/attachments/12888">Snip20141227_1.png</a> <a class="icon-only icon-download" title="Download" href="/attachments/download/12888/Snip20141227_1.png">Snip20141227_1.png</a> added</li></ul><p>Felix Schäfer wrote:</p>
<blockquote>
<p>Krishna Gollamudi wrote:</p>
<blockquote>
<p>Jean, Are you able to retain tables coming from the incoming emails?</p>
</blockquote>
<p>If you mean html tables: no, this method transforms the html-part of a text-part-less email to a somewhat equivalent text, but does not know about textile or markdown markup that might be used to render that text.</p>
</blockquote>
<p>Felix, I mean if the incoming email has data like the attached, then the issue is getting created just with the text in separate line.</p> Redmine - Feature #16962: Better handle html-only emailshttps://www.redmine.org/issues/16962?journal_id=606232014-12-29T10:06:16ZFelix Schäfer
<ul></ul><p>Krishna Gollamudi wrote:</p>
<blockquote>
<p>Felix, I mean if the incoming email has data like the attached, then the issue is getting created just with the text in separate line.</p>
</blockquote>
<p>I guess that would be the case, but I'd have to test that. Could you either attach to this issue or send me (my email address should be on my user page) if you'd rather not have it accessible here the source of an email (in .eml format) with such a table so that I can test it?</p> Redmine - Feature #16962: Better handle html-only emailshttps://www.redmine.org/issues/16962?journal_id=606262014-12-29T16:39:47ZKrishna Gollamudi
<ul><li><strong>File</strong> <a href="/attachments/12894">Test Email.msg</a> <a class="icon-only icon-download" title="Download" href="/attachments/download/12894/Test%20Email.msg">Test Email.msg</a> added</li></ul><p>Felix Schäfer wrote:</p>
<blockquote>
<p>Krishna Gollamudi wrote:</p>
<blockquote>
<p>Felix, I mean if the incoming email has data like the attached, then the issue is getting created just with the text in separate line.</p>
</blockquote>
<p>I guess that would be the case, but I'd have to test that. Could you either attach to this issue or send me (my email address should be on my user page) if you'd rather not have it accessible here the source of an email (in .eml format) with such a table so that I can test it?</p>
</blockquote>
<p>No problem Felix. Here is the email in .msg format straight from my outlook account.</p> Redmine - Feature #16962: Better handle html-only emailshttps://www.redmine.org/issues/16962?journal_id=606952015-01-05T22:49:57ZKrishna Gollamudi
<ul></ul><p>Got any update Felix??</p> Redmine - Feature #16962: Better handle html-only emailshttps://www.redmine.org/issues/16962?journal_id=607502015-01-09T13:31:24ZFelix Schäfer
<ul></ul><p>Krishna, I had first to transform your .msg file to .eml format, I'm not sure how good the converters are, but I was able to see the same content as is visible in your screenshot.</p>
<p>The result from the .eml file is:</p>
<pre>\n\n \n\nTeam project: \n\nTe\n\nArea: \n\n\\\\Te\\\\OH 30\n\nIteration: \n\n\\\\Te\\\\OH 30\\\\Iteration 1\n\nAssigned to: \n\nKP\n\nState: \n\nNew\n\nReason: \n\nNew\n\nChanged by: \n\nGo\n\nChanged date: \n\n12/19/2014 3:21:02 PM\n\nChanged fields \n\nField \n\nNew Value \n\nHistory\n\nPer triage QA confirmed this is happening when user trying to see history.\n\n \n\nField \n\nNew Value \n\nOld Value \n\nTriage\n\nNeed More Info - Dev\n\nNeed More Info - QA\n\nAssigned To\n\nKP\n\nGo\n\n \n\n</pre>
<p>which basically means that every table cell gets thrown on its own line. The first few lines would look like:</p>
<p><code>START</code></p>
<p>Team project:</p>
<p>Te</p>
<p>Area:</p>
<p>\\Te\\OH 30</p>
<p>Iteration:</p>
<p>\\Te\\OH 30\\Iteration 1</p>
<p>Assigned to:</p>
<p>KP<br /><code>END</code></p>
<p>With the current method that would be like:</p>
<p><code>START</code><br /> Team project: TeArea: \\Te\\OH 30Iteration: \\Te\\OH 30\\Iteration 1Assigned to: KP<br /><code>END</code></p>
<p>Please note that the converted .eml had a text part, so the patch proposed here would not change how this email is imported to Redmine, as the text part has precedence over the html part in any case.</p> Redmine - Feature #16962: Better handle html-only emailshttps://www.redmine.org/issues/16962?journal_id=607512015-01-09T13:40:15ZFelix Schäfer
<ul></ul><p>Jean-Philippe Lang wrote:</p>
<blockquote>
<p>A simple fix could be <code>gsub(/\s+/, ' ')</code> the html before processing it with Loofah.</p>
</blockquote>
<p>I though about it a bit more and that would also mangle whitespace in places where it is significant in html, for example in <code>< pre></code> blocks. For whitespace that wouldn't be a problem as it would be just text in textile, where any number of whitespace will just get collapse in the rendered html afterwards, but <code>< pre>foo\nbar\nbaz</pre></code> would then be changed to <code>< pre>foo bar baz</pre></code> and converted to <code>\nfoo bar baz\n</code> instead of <code>\nfoo\nbar\nbaz\n</code>.</p>
<p>I have looked for a nicer solution with loofah or nokogiri but couldn't find one yet. I have <a href="https://github.com/flavorjones/loofah/issues/82" class="external">asked on the loofah tracker</a> for advice on this matter.</p> Redmine - Feature #16962: Better handle html-only emailshttps://www.redmine.org/issues/16962?journal_id=634532015-04-29T13:59:40ZToshi MARUYAMA
<ul><li><strong>Related to</strong> <i><a class="issue tracker-1 status-5 priority-5 priority-high2 closed" href="/issues/19737">Defect #19737</a>: HTML Sanitizer not working for Outlook mails</i> added</li></ul> Redmine - Feature #16962: Better handle html-only emailshttps://www.redmine.org/issues/16962?journal_id=634582015-04-29T15:10:20ZToshi MARUYAMA
<ul><li><strong>Related to</strong> <i><a class="issue tracker-1 status-5 priority-4 priority-default closed" href="/issues/19740">Defect #19740</a>: "Truncate emails after one of these lines" setting is not working</i> added</li></ul> Redmine - Feature #16962: Better handle html-only emailshttps://www.redmine.org/issues/16962?journal_id=642062015-06-15T21:19:58ZJean-Philippe Langjp_lang@yahoo.fr
<ul><li><strong>Status</strong> changed from <i>Needs feedback</i> to <i>Closed</i></li><li><strong>Assignee</strong> set to <i>Jean-Philippe Lang</i></li><li><strong>Target version</strong> changed from <i>Candidate for next major release</i> to <i>3.1.0</i></li><li><strong>Resolution</strong> set to <i>Fixed</i></li></ul><p>Loofah is now used for processing HTML-only emails (<a class="changeset" title="Better handle html-only emails (#16962)." href="https://www.redmine.org/projects/redmine/repository/svn/revisions/14313">r14313</a>). HTML processing was moved from the MailHandler to the text formatter so we can convert it to wiki syntax.</p> Redmine - Feature #16962: Better handle html-only emailshttps://www.redmine.org/issues/16962?journal_id=642112015-06-15T23:54:05ZDeoren Moor
<ul></ul><p>Jean-Philippe Lang wrote:</p>
<blockquote>
<p>Loofah is now used for processing HTML-only emails (<a class="changeset" title="Better handle html-only emails (#16962)." href="https://www.redmine.org/projects/redmine/repository/svn/revisions/14313">r14313</a>). HTML processing was moved from the MailHandler to the text formatter so we can convert it to wiki syntax.</p>
</blockquote>
<p>Should the changes have also solved processing issues with Apple Mail emails (<a class="issue tracker-1 status-5 priority-4 priority-default closed" title="Defect: Scraped emails include CSS from HTML emails (Closed)" href="https://www.redmine.org/issues/15716">#15716</a>) from Mac OS X and iOS?</p> Redmine - Feature #16962: Better handle html-only emailshttps://www.redmine.org/issues/16962?journal_id=899072019-02-09T07:12:40ZGo MAEDA
<ul><li><strong>Related to</strong> <i><a class="issue tracker-1 status-5 priority-4 priority-default closed" href="/issues/10201">Defect #10201</a>: MailHandler ignores line feeds when handling html-only emails</i> added</li></ul> Redmine - Feature #16962: Better handle html-only emailshttps://www.redmine.org/issues/16962?journal_id=899202019-02-09T09:08:02ZGo MAEDA
<ul><li><strong>Related to</strong> <i><a class="issue tracker-1 status-5 priority-4 priority-default closed" href="/issues/13284">Defect #13284</a>: Should not try to strip HTML tags from text/plain MIME part</i> added</li></ul> Redmine - Feature #16962: Better handle html-only emailshttps://www.redmine.org/issues/16962?journal_id=899532019-02-10T04:15:43ZGo MAEDA
<ul><li><strong>Related to</strong> <i><a class="issue tracker-1 status-5 priority-4 priority-default closed" href="/issues/6705">Defect #6705</a>: Incoming emails from Apple Mail not parsed correctly</i> added</li></ul> Redmine - Feature #16962: Better handle html-only emailshttps://www.redmine.org/issues/16962?journal_id=899572019-02-10T04:41:22ZGo MAEDA
<ul><li><strong>Related to</strong> <i><a class="issue tracker-3 status-5 priority-4 priority-default closed" href="/issues/12937">Patch #12937</a>: mail_handler patch</i> added</li></ul> Redmine - Feature #16962: Better handle html-only emailshttps://www.redmine.org/issues/16962?journal_id=899622019-02-10T05:19:26ZGo MAEDA
<ul><li><strong>Related to</strong> <i><a class="issue tracker-1 status-5 priority-4 priority-default closed" href="/issues/14668">Defect #14668</a>: Incoming emails has wrong line breaks</i> added</li></ul> Redmine - Feature #16962: Better handle html-only emailshttps://www.redmine.org/issues/16962?journal_id=902622019-02-19T02:10:25ZGo MAEDA
<ul><li><strong>Related to</strong> <i><a class="issue tracker-2 status-5 priority-4 priority-default closed" href="/issues/13844">Feature #13844</a>: Option to enable HTML emails for issue creation.</i> added</li></ul>