Defect #14009
closedIssue Editing/Viewing - ASCII-8BIT/UTF-8 Problems
Added by Roger Griffiths over 11 years ago. Updated 10 months ago.
0%
Description
- OS: openSUSE 12.3
- Redmine: 2.3.0
- Rails: 3.2.13
- Ruby: 1.9.3p392
- Database: MariaDB 5.5.29
- ActionView::TemplateError (incompatible character encodings: ASCII-8BIT and UTF-8)
- app/views/issues/show.html.erb:120
I am unable to access a ticket after entering an update to the ticket including the following characters:-
- Superscript Two - U+00B2
- Greek Small Letter Lambda - U+03BB
- Greek Small Letter Pi - U+03C0
- Greek Small Letter Sigma - U+03C3
I received an e-mail from Redmine indicating that these values had been accepted by the system as entered however I am no longer able to view the ticket view/edit the Redmine interface.
Updated by Jean-Baptiste Barth over 11 years ago
- Status changed from New to Needs feedback
Can you show us a more complete stack trace ? The mentionned line is just a reference to an other view, maybe the full trace will include more informations about where the problem is so we can investigate better.
Updated by Roger Griffiths over 11 years ago
Updated by Roger Griffiths over 11 years ago
Problem is that I have to type the stack trace, so it gets awkward.
Updated by Jean-Baptiste Barth over 11 years ago
OK. Can you see SubmittingBugs and give us the requested informations about your redmine instance (I'm especially interested in your "mysql" gem version, and your database encoding if you have it).
Updated by Roger Griffiths over 11 years ago
- app/controllers/issues_controller.rb:119
- app/controllers/issues_controller.rb:116
Updated by Roger Griffiths over 11 years ago
Database was created using UTF-8 as per your Wiki.
Gems will be as per normal for openSUSE 12.3 (updated about 1-2wks ago) or that installed by redmine-2.3.0.
According to: gem list MySQL- mysql (2.9.0, 2.8.1)
- mysql2 (0.3.11)
Updated by Roger Griffiths over 11 years ago
See original report above for all information explicitly mentioned in SubmittingBugs (except the complete stack trace which has been supplemented as requested).
Updated by Roger Griffiths over 11 years ago
- Superscript Two - \302\262
- Greek Small Letter Lambda - \316\273
- Greek Small Letter Pi - \317\200
- Greek Small Letter Sigma - \317\203
Updated by Roger Griffiths over 11 years ago
Worked around issue by replacing UTF-8 characters with pure ASCII alternatives.
This should NOT be an issue with a system that accepts and even promotes usage of UTF-8 characters.
Updated by Toshi MARUYAMA over 11 years ago
Check your database encoding is UTF-8.
See:
http://www.redmine.org/boards/1/topics/36573?r=36629#message-36629
Updated by Roger Griffiths over 11 years ago
Database encoding is confirmed as UTF-8.
The encoding previously mentioned was in an ASCII encoded SQL dump file.
When I manually updated the database I used the raw Unicode characters in my SQL statements.
Updated by Toshi MARUYAMA over 11 years ago
Roger Griffiths wrote:
Database encoding is confirmed as UTF-8.
The encoding previously mentioned was in an ASCII encoded SQL dump file.
When I manually updated the database I used the raw Unicode characters in my SQL statements.
Have you resolved the problem?
Updated by Roger Griffiths over 11 years ago
No - I have worked around it with a kludge.
The reported issue still needs to be resolved.
Updated by Toshi MARUYAMA over 11 years ago
Did you check "show session variables like 'char%'" and "show global variables like 'char%';"?
http://www.redmine.org/boards/1/topics/36573?r=36624#message-36624
Why do you use MariaDB?
Is MariaDB compatible with MySQL client (Ruby mysql2 adapter)?
Updated by Roger Griffiths over 11 years ago
MariaDB is the default database installation for openSUSE 12.3 and is a compatible enhancement of MySQL (i.e. a drop in replacement).
I am a programmer myself and have double checked my installation before reporting the issue here. I hate it when users report problems without doing such checks themselves first.
The database is running fine and processing UTF-8 correctly, the Ruby code would appear to be at fault based on my limited investigation (I do not have the time to look into it and fix it myself).
Updated by Roger Griffiths over 11 years ago
As the issue does not appear to occur with the main Redmine server what component/software versions are being used for THIS site.
Updated by Jan Niggemann (redmine.org team member) about 11 years ago
Toshi MARUYAMA wrote:
Is MariaDB compatible with MySQL client (Ruby mysql2 adapter)?
MariaDB is a drop-in replacement for MySQL, it's a fork of MySQL created by its own inventor (Monty widenius) after he left Oracle. There shouldn't be any issues using MariaDB instead of MySQL, except that the gem needs to be compatible with the db-client-lib version.
Updated by Domingo Galdos almost 11 years ago
We had the same issue using the current latest stable redmine version.
We have a patch to fix it (below)
I had the same issue with Redmine, after upgrading from ruby 1.8.x to ruby 2.0.x. It turned out that indeed, an unrecognized character had been pasted in by a user (in our case it was ""), and the GUI allows this even though it causes this issue. Correcting the field in the database fixes it.
A more permanent fix is to follow the advice from http://stackoverflow.com/a/5233939/368503 and add an initializer that monkeypatches the mysql lib to automatically re-encode problematic strings so the issue never hits Rails or Redmine. I tested this in our configuration successfully. Our configuration is:
- ruby 2.0.0-p353 (2013-11-22) [x86_64-linux]
- redmine 2.4.1.stable
- passenger 4.0.30
- bundler 1.2.4
- rails 3.2.15
- Database: mysqld Ver 5.0.77 for redhat-linux-gnu on x86_64 (Source distribution)
Let me know if any more information is needed to vet this before moving forward or anything else I should be aware of as this is my first time contributing bugfixes back upstream to Redmine.
Updated by Domingo Galdos almost 11 years ago
Turns out switching to the mysql2 adapter (instead of the mysql adapter) by changing one line of database.yml is a much simpler and more complete fix. So disregard the solution mentioned above.
Nevertheless, given that the newer version combinations of Redmine/dependencies work so badly with the mysql adapter, perhaps a deprecation warning could be added? Would have saved me a bunch of time, and I'm neither the first nor last to run into this issue. Does that sound reasonable, or some other such solution to avoid anyone wasting more than a couple minutes on this issue again? Thanks!