Defect #20143
open
Mailhandler cannot handle 4-byte characters
0%
Description
When receiving an e-mail containing a 4-byte character (an apple-emoji in the example), the mail-handler dies with
Mysql2::Error: Incorrect string value: '\xF0\x9F\x98\x89' for column 'description' at row 1: INSERT INTO `issues` (`project_id`, `author_id`, `priority_id`, `tracker_id`, `status_id`, `subject`, `description`, `start_date`, `updated_on`, `created_on`, `lock_version`) VALUES (15, 4, 2, 3, 1, 'Emoticon test', 'Here comes an emoji: ?~_~X~I', '2015-06-21', '2015-06-21 18:05:05', '2015-06-21 18:05:05', 0)
and no ticket is created.
Related issues
Updated by Toshi MARUYAMA over 9 years ago
- Category changed from Email receiving to Database
Updated by Toshi MARUYAMA over 9 years ago
- Related to Patch #19742: RedmineInstall: MySQL: collation_database added
Updated by Toshi MARUYAMA over 9 years ago
- Has duplicate Defect #10772: 4-byte utf-8 characters added
Updated by Ragnar Nevries over 9 years ago
Toshi MARUYAMA wrote:
Try utf8mb4 (#10772#note-7).
In fact, that solves the problem, but it does not work for all tables:
MariaDB [redmine]> ALTER TABLE changesets CONVERT TO CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;
ERROR 1071 (42000): Specified key was too long; max key length is 767 bytes
MariaDB [redmine]> ALTER TABLE schema_migrations CONVERT TO CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;
ERROR 1071 (42000): Specified key was too long; max key length is 767 bytes
Wouldn't it be great to have this character encoding used per default or as an option for mysql? Four byte unicode characters are not that rare these days, especially in emails (where the emails client converts ASCII-things to utf8). Sorry if this is already possible, I'm using the great sameersbn Dockerfile for installing/running redmine.
Updated by Toshi MARUYAMA over 9 years ago
Ragnar Nevries wrote:
Wouldn't it be great to have this character encoding used per default or as an option for mysql?
Redmine does not define creating database process.
I don't prefer defining MySQL encoding (#19742#note-3).
I'm using the great sameersbn Dockerfile for installing/running redmine.
Please ask its maintainer.
Updated by Peter Pfläging about 9 years ago
Ragnar Nevries wrote:
Toshi MARUYAMA wrote:
Try utf8mb4 (#10772#note-7).
In fact, that solves the problem, but it does not work for all tables:
MariaDB [redmine]> ALTER TABLE changesets CONVERT TO CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;
ERROR 1071 (42000): Specified key was too long; max key length is 767 bytesMariaDB [redmine]> ALTER TABLE schema_migrations CONVERT TO CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;
ERROR 1071 (42000): Specified key was too long; max key length is 767 bytesWouldn't it be great to have this character encoding used per default or as an option for mysql? Four byte unicode characters are not that rare these days, especially in emails (where the emails client converts ASCII-things to utf8). Sorry if this is already possible, I'm using the great sameersbn Dockerfile for installing/running redline.
It works for all tables, if you change the ROW_FORMAT in MySQL/MariaDB to DYNAMIC!
I've made a script for that http://www.pflaeging.net/blog/archives/938 (Sorry, it's in german, but the script ist in sh ;-))
Updated by Marius BĂLTEANU over 5 years ago
- Related to Feature #31921: Changes to properly support 4 byte characters (emoji) when database is MySQL added