Project

General

Profile

Actions
Copy link

Defect #24616

closed

Should not replace all invalid utf8 characters (e.g in mail)

Added by Pavel Rosický almost 8 years ago. Updated almost 8 years ago.

Status:
Closed
Priority:
Normal
Assignee:
-
Category:
I18n
Target version:
3.4.0
Start date:
Due date:
% Done:

0%

Estimated time:
Resolution:
Fixed
Affected version:

Description

Hello,
I've an email, that is encoded in utf8, but it contains an invalid character. In this case, redmine converts the content to us-ascii and then to utf8. This step will replace non-ascii compatible chars to "?". Why?

1) Failure:
MailHandlerTest#test_invalid_utf8 [/test/unit/mail_handler_test.rb:548]:
Expected: "Здравствуйте?" 
  Actual: "?????????????"

I changed Redmine::CodesetUtil.replace_invalid_utf8(str) and Redmine::CodesetUtil.to_utf8(str, encoding)

        str = str.encode("US-ASCII", :invalid => :replace, :undef => :replace, :replace => '?').encode("UTF-8")

to 
        str = str.encode("UTF-8", :invalid => :replace, :undef => :replace, :replace => '?')

all tests are passing with this change. 
  Redmine version                3.3.1.stable
  Ruby version                   2.1.5-p273 (2014-11-13) [x64-mingw32]
  Rails version                  4.2.7.1
  Environment                    production
  Database adapter               Mysql2
SCM:
  Git                            2.10.1
  Filesystem
Redmine plugins:
  no plugin installed

Files

Download all files
invalid_utf8_test.patch (1.49 KB) invalid_utf8_test.patch spec Pavel Rosický, 2016-12-15 01:42
defect-24616.diff (1.73 KB) defect-24616.diff fix + tests (generated from Pavel Rosický's contribution) Go MAEDA, 2016-12-29 07:52
Actions
Copy link

Also available in: Atom PDF