https://www.redmine.org/https://www.redmine.org/favicon.ico?16793021292009-08-13T13:06:48ZRedmineRedmine - Defect #3717: Swedish characters get replaced with kanji signs in an issue when added via emailhttps://www.redmine.org/issues/3717?journal_id=105362009-08-13T13:06:48ZArnaud Lesauvage
<ul></ul><p>I can confirm this also happens on my Windows box.<br />Database is in PostgreSQL 8.3 (UTF8 encoding).<br />Email messages are sent in ISO-8859-1.<br />Messages are fetched via IMAP.<br />The mail server is running Zimbra 4.5.13.</p>
<p>The problematic headers were :<br />Subject: test =?ISO-8859-1?Q?cr=E9ation_t=E2che_email?=<br />Content-Type: text/plain; charset=ISO-8859-1; format=flowed<br />Content-Transfer-Encoding: 7bit</p>
<p>Resulting task name was "test cr辿ation t但che email".<br />Same platform info as the original poster, appart from Win2003-R2 OS and PostgreSQL 8.3 database.</p> Redmine - Defect #3717: Swedish characters get replaced with kanji signs in an issue when added via emailhttps://www.redmine.org/issues/3717?journal_id=106072009-08-19T14:40:15ZFredrik Frodlund
<ul></ul><p>Arnaud Lesauvage wrote:</p>
<blockquote>
<p>I can confirm this also happens on my Windows box.</p>
</blockquote>
<p>Does your problem also temporarily disappear after restarting the Ruby application?</p> Redmine - Defect #3717: Swedish characters get replaced with kanji signs in an issue when added via emailhttps://www.redmine.org/issues/3717?journal_id=106792009-08-25T10:12:07ZArnaud Lesauvage
<ul></ul><p>Fredrik Frodlund wrote:</p>
<blockquote>
<p>Does your problem also temporarily disappear after restarting the Ruby application?</p>
</blockquote>
<p>I always have the problem, but I don't understand what you mean by restarting the ruby application.<br />In my setup (IIS+fastcgi) I have one ruby.exe process that manages web requests and a scheduled task that launches a ruby.exe process to fetch emails.<br />This scheduled task reloads the environment every time it runs (very anoying since it takes ~20 seconds of processing time).<br />I could stop/restart the web service to recycle the fastcgi ruby process but that wouldn't change anything, right ?</p> Redmine - Defect #3717: Swedish characters get replaced with kanji signs in an issue when added via emailhttps://www.redmine.org/issues/3717?journal_id=107272009-08-31T06:34:38ZFredrik Frodlund
<ul></ul><p>Arnaud Lesauvage wrote:</p>
<blockquote>
<p>Fredrik Frodlund wrote:</p>
<blockquote>
<p>Does your problem also temporarily disappear after restarting the Ruby application?</p>
</blockquote>
<p>I always have the problem, but I don't understand what you mean by restarting the ruby application.<br />In my setup (IIS+fastcgi) I have one ruby.exe process that manages web requests and a scheduled task that launches a ruby.exe process to fetch emails.<br />This scheduled task reloads the environment every time it runs (very anoying since it takes ~20 seconds of processing time).<br />I could stop/restart the web service to recycle the fastcgi ruby process but that wouldn't change anything, right ?</p>
</blockquote>
<p>I guess since you have a differing platform, that does not apply here... Nevermind. :)</p> Redmine - Defect #3717: Swedish characters get replaced with kanji signs in an issue when added via emailhttps://www.redmine.org/issues/3717?journal_id=107282009-08-31T06:56:05ZArnaud Lesauvage
<ul></ul><p>I just reinstalled everything on Archlinux.<br />Redmine is from trunk, revision 2847. Mongrel<br />Exact same problem.</p>
<p>I stopped/restarted mongrel to see whether it helped, but it doesn't.</p> Redmine - Defect #3717: Swedish characters get replaced with kanji signs in an issue when added via emailhttps://www.redmine.org/issues/3717?journal_id=107962009-09-02T18:38:55ZAndré H.
<ul></ul><p>Same problem here. German umlauts (Ä, Ö, Ü) are turned into kanji signs.<br />Redmine runs on a linux box here. Everything is UTF-8. Only the the mail's <strong>subject</strong> is inserted with wrong encoding/characters. Everything works fine with message body.</p> Redmine - Defect #3717: Swedish characters get replaced with kanji signs in an issue when added via emailhttps://www.redmine.org/issues/3717?journal_id=108152009-09-03T15:43:00ZFredrik Frodlund
<ul></ul><p>I just ran into a weird thing. I tried sending a mail with just "åäö ÅÄÖ" in the subject and as expected, the characters got converted to kanji. However, I tried adding spaces between each character, like this: "å ä ö Å Ä Ö" and the characters were left intact!</p>
<p>I sure hope this might be a clue as to why it happens.</p> Redmine - Defect #3717: Swedish characters get replaced with kanji signs in an issue when added via emailhttps://www.redmine.org/issues/3717?journal_id=108242009-09-04T06:07:32ZAndré H.
<ul></ul><p>I also tested something.<br />Try to put "Ä öl, häuser, für" as subject. No character will be converted into strange signs. It will suffice to put an ÄÖÅ (whatever) as first character of the subject and the rest of the subject will be fine.</p>
<p>Perhaps this is another hint.</p> Redmine - Defect #3717: Swedish characters get replaced with kanji signs in an issue when added via emailhttps://www.redmine.org/issues/3717?journal_id=108282009-09-04T08:56:07ZFredrik Frodlund
<ul></ul><p>I just noted that mails sent in other encodings than UTF8 will also get the "kanji"-treatment. I can also confirm André's test.</p> Redmine - Defect #3717: Swedish characters get replaced with kanji signs in an issue when added via emailhttps://www.redmine.org/issues/3717?journal_id=108292009-09-04T09:00:17ZArnaud Lesauvage
<ul></ul><p>Fredrik Frodlund wrote:</p>
<blockquote>
<p>I just noted that mails sent in other encodings than UTF8 will also get the "kanji"-treatment. I can also confirm André's test.</p>
</blockquote>
<p>Indeed, see <a href="http://www.redmine.org/issues/3717#note-1" class="external">my comment</a> , encoding was ISO-8859-1.</p> Redmine - Defect #3717: Swedish characters get replaced with kanji signs in an issue when added via emailhttps://www.redmine.org/issues/3717?journal_id=112952009-10-07T15:58:48ZIgnacio Carrera
<ul></ul><p>It got broken in <a class="issue tracker-3 status-5 priority-4 priority-default closed" title="Patch: broken subject when submitting issue via email written in japanese (Closed)" href="https://www.redmine.org/issues/2059">#2059</a> (<a href="http://www.redmine.org/projects/redmine/repository/revisions/1990/diff/trunk/app/models/mail_handler.rb" class="external">see patch</a>)</p>
<p>The culprit: /app/models/mail_handler.rb:123</p>
<pre>
issue.subject = email.subject.chomp.toutf8
</pre>
<p>TMail#subject handles my case correctly (subject in iso-8859-1, with Spanish characters), and I suspect most other cases (as it should).</p>
<p>I don't know any Japanese, so all I can do is ask someone who does to verify this:</p>
<pre>
TMail::Unquoter.unquote_and_convert_to("=?ISO-8859-1?Q?Nueva_modificaci=F3n?=", "utf-8")
# => "Nueva modificación" (this is correct)
TMail::Unquoter.unquote_and_convert_to("=?iso-2022-jp?B?GyRCJCIkJCQmJCgkKhsoQg==?=", "utf-8")
# => "あいうえお" (this seems OK to me, but I'm no Japanese)
</pre>
<p>If the result is incorrect, it should be fixed (or monkey patched) in TMail::Unquoter, so as not to break other encodings.</p>
<p>Please notice you may need to require tmail or something.</p>
<p>But I believe that is indeed correct, so the problem may be elsewhere (don't know where though).</p>
<p>-- nachokb</p> Redmine - Defect #3717: Swedish characters get replaced with kanji signs in an issue when added via emailhttps://www.redmine.org/issues/3717?journal_id=117162009-10-30T10:08:06ZFredrik Frodlund
<ul></ul><p>Now that we know what the culprit is, shouldn't we be able to resolve this. I'm just getting a little tired of fixing issues via directly in the database, is all. :)</p> Redmine - Defect #3717: Swedish characters get replaced with kanji signs in an issue when added via emailhttps://www.redmine.org/issues/3717?journal_id=117212009-10-30T14:41:37ZIgnacio Carrera
<ul></ul><p>Fredrik Frodlund wrote:</p>
<blockquote>
<p>Now that we know what the culprit is, shouldn't we be able to resolve this. I'm just getting a little tired of fixing issues via directly in the database, is all. :)</p>
</blockquote>
<p>Jean Philippe fixed it in a branch: <a class="external" href="http://www.redmine.org/projects/redmine/repository/revisions/2620/diff/sandbox/rails-2.3/app/models/mail_handler.rb">http://www.redmine.org/projects/redmine/repository/revisions/2620/diff/sandbox/rails-2.3/app/models/mail_handler.rb</a> (though I don't know if it will get to a release soon)...</p>
<p>-- nachokb</p> Redmine - Defect #3717: Swedish characters get replaced with kanji signs in an issue when added via emailhttps://www.redmine.org/issues/3717?journal_id=117782009-11-03T13:52:26ZFredrik Frodlund
<ul></ul><p>The code is looking a bit different in <a class="changeset" title="Fixed: HTML entities displayed when editing an issue note (#3996)." href="https://www.redmine.org/projects/redmine/repository/svn/revisions/2924">r2924</a>. I would like at least to make a local change to make the problem go away. Anyone that can help me with this?</p>
<p>/app/models/mail_handler.rb:124<br /><pre>
issue.subject = issue.subject.toutf8 if issue.subject.respond_to?(:toutf8)
</pre></p> Redmine - Defect #3717: Swedish characters get replaced with kanji signs in an issue when added via emailhttps://www.redmine.org/issues/3717?journal_id=117792009-11-03T14:24:06ZIgnacio Carrera
<ul></ul><p>Fredrik Frodlund wrote:</p>
<blockquote>
<p>The code is looking a bit different in <a class="changeset" title="Fixed: HTML entities displayed when editing an issue note (#3996)." href="https://www.redmine.org/projects/redmine/repository/svn/revisions/2924">r2924</a>. I would like at least to make a local change to make the problem go away. Anyone that can help me with this?</p>
<p>/app/models/mail_handler.rb:124<br />[...]</p>
</blockquote>
<p>Just comment that line out...</p>
<p>-- nachokb</p> Redmine - Defect #3717: Swedish characters get replaced with kanji signs in an issue when added via emailhttps://www.redmine.org/issues/3717?journal_id=117802009-11-03T14:50:46ZFredrik Frodlund
<ul></ul><p>Wow, it really was that simple. Thanks! :)</p> Redmine - Defect #3717: Swedish characters get replaced with kanji signs in an issue when added via emailhttps://www.redmine.org/issues/3717?journal_id=132672009-12-26T12:08:15ZJean-Philippe Langjp_lang@yahoo.fr
<ul><li><strong>Status</strong> changed from <i>New</i> to <i>Resolved</i></li><li><strong>Target version</strong> set to <i>0.9.0</i></li><li><strong>Resolution</strong> set to <i>Fixed</i></li></ul><p>Fix committed in <a class="changeset" title="Fixed: subject of incoming emails gets mangled (#3717)." href="https://www.redmine.org/projects/redmine/repository/svn/revisions/3247">r3247</a>.</p> Redmine - Defect #3717: Swedish characters get replaced with kanji signs in an issue when added via emailhttps://www.redmine.org/issues/3717?journal_id=132812009-12-26T13:48:18ZJean-Philippe Langjp_lang@yahoo.fr
<ul><li><strong>Status</strong> changed from <i>Resolved</i> to <i>Closed</i></li></ul><p>Merged in 0.9-stable in <a class="changeset" title="Merged r3246 and r3247 from trunk." href="https://www.redmine.org/projects/redmine/repository/svn/revisions/3248">r3248</a>.</p> Redmine - Defect #3717: Swedish characters get replaced with kanji signs in an issue when added via emailhttps://www.redmine.org/issues/3717?journal_id=132932009-12-26T17:22:41ZIgnacio Carrera
<ul></ul><p>Jean-Philippe Lang wrote:</p>
<blockquote>
<p>Fix committed in <a class="changeset" title="Fixed: subject of incoming emails gets mangled (#3717)." href="https://www.redmine.org/projects/redmine/repository/svn/revisions/3247">r3247</a>.</p>
</blockquote>
<p>Thanks!</p>