Project

General

Profile

Actions
Copy link

Feature #26423

open

Character encoding auto-detection using rchardet

Added by Dmitri Zhabinski over 7 years ago. Updated over 7 years ago.

Status:
New
Priority:
Normal
Assignee:
-
Category:
I18n
Target version:
-
Start date:
Due date:
% Done:

0%

Estimated time:
Resolution:

Description

Current character encoding auto-detection cant differentiate cp1251 from cp866 (if both encodings are configured).

But https://rubygems.org/gems/rchardet/ can !

I have made proof of concept patch on my Redmine 3.3.1 from debian package.
And it just works without significant performance impact.

Files

rchardet.patch (603 Bytes) rchardet.patch Dmitri Zhabinski, 2017-07-13 11:05
Actions
Copy link
 #1

Updated by Toshi MARUYAMA over 7 years ago

  • Category set to I18n
Actions
Copy link
 #2

Updated by Toshi MARUYAMA over 7 years ago

Could you add tests?
source:tags/3.4.1/test/unit/lib/redmine/codeset_util_test.rb

I don't like auto detection.
Japanese has many character encodings (UTF-8, EUC-JP, CP932, Shift-JIS, and ISO-2022-JP).
Auto detection misbehaves in many cases in Japanese.

I think Redmine should have option.
  • No auto detection
  • unshift
  • push
Actions
Copy link

Also available in: Atom PDF