Project

General

Profile

Actions

Defect #55

closed

Non-unicode repository content not displayed correctly

Added by Nikolay Solakov almost 17 years ago. Updated over 16 years ago.

Status:
Closed
Priority:
Normal
Assignee:
-
Category:
-
Target version:
-
Start date:
Due date:
% Done:

0%

Estimated time:
Resolution:
Affected version:

Description

Hi,
Look at the attached screenshot.

When I set the page encoding to Windows-1251 it shows the characters correctly.

Thanks,
Nikolay


Files

cyrillic_repo_files.JPG (134 KB) cyrillic_repo_files.JPG Nikolay Solakov, 2008-02-03 10:50
Actions #1

Updated by Jean-Philippe Lang almost 17 years ago

As files in the repository may be encoded to anything, I don't
see any solution that would work in any cases.
Do you have a proposition for this ?

Actions #2

Updated by Nikolay Solakov almost 17 years ago

Unfortunately I don't :(
Maybe there should have some sort of auto-detecting feature
to say what is the encoding of the files, but I don't know
how it is implemented for example in Firefox's
View->Encoding.

Thanks,
Nikolay

Actions #3

Updated by Kyosuke Takayama almost 17 years ago

Hi.

It should use iconv.

I wrote patch for svn viewer.
It is optimized for Japanese now. However, there is not influence
towards the English zone.
And It can cope with other character codes zone if you change
CharSets.

Index: app/models/svn_repos.rb =================================================================

--- app/models/svn_repos.rb (revision 537)
+++ app/models/svn_repos.rb (working copy)
@ -17,7 +17,23 @

require 'rexml/document'
require 'cgi'
+require 'iconv'

class Iconv
CharSets = ['iso-2022-jp', 'euc-jp', 'utf-8', 'shift_jis',
'windows-31j']

def self.to_utf8(str)
+ return str if /\A[\r\n\t\x20-\x7e]*\Z/n.match(str) # for
us-ascii
+ CharSets.each do |char|
+ begin
+ return Iconv.conv("UTF-8", char, str)
+ rescue Iconv::IllegalSequence
+ end
+ end
+ str
+ end
end

module SvnRepos

class CommandFailed < StandardError #:nodoc:
@ -178,7 +194,7 @
cat = io.read
end
return nil if $? && $?.exitstatus != 0
- cat
+ Iconv.to_utf8(cat)
rescue Errno::ENOENT => e
raise CommandFailed
end
=============================================================== ===========
  • for Cyrillic support.
    CharSets = ['windows-1251', 'utf-8']

Thank you.

Actions #4

Updated by Nikolay Solakov almost 17 years ago

Hi,
It's not working for me. I put the
CharSets = ['windows-1251', 'utf-8']
instead of
CharSets = ['iso-2022-jp', 'euc-jp', 'utf-8', 'shift_jis',
'windows-31j']
The result is the same. :(

Thanks,
Nikolay

Actions #5

Updated by Kyosuke Takayama almost 17 years ago

It does work only in FILE view.
But does not work in DIFF view.

Probably, in the case of DIFF, it is so..

Index: app/models/svn_repos.rb =================================================================
--- app/models/svn_repos.rb (revision 538)
+++ app/models/svn_repos.rb (working copy)
@ -310,7 +326,7 @
self << diff_table if diff_table.length

1

diff_table = DiffTable.new type
end
- a = diff_table.add_line line
+ a = diff_table.add_line(Iconv.to_utf8(line))
end
self << diff_table
end
Actions #6

Updated by Nikolay Solakov almost 17 years ago

Thanks,
I can now see the cyrillic chars.

Hope Jean-Philippe integrate it soon.

Nikolay.

Actions #7

Updated by Jean-Philippe Lang almost 17 years ago

Thanks for your help.

I propose to add the list of possible charsets in lang files,
so that they can be set for each language.
The charsets defined for the default language of your redmine
installation would be used for iconv convertion.

Actions #8

Updated by Jean-Philippe Lang almost 17 years ago

One more thing.
It should be ok for diffs, but trying to convert files to utf-8
(in cat method) in any case may cause some problems (eg. binary
files). It should be done only for text files.
I'll have to add a check.

Actions #9

Updated by Nikolay Solakov over 16 years ago

Hi,
Is this bug-feature implemented?
When I updated my redMine recently the incorrect characters
appeared again :(

Thank,
Nikolay

Actions #10

Updated by Nikolay Solakov over 16 years ago

Forgot to say that I was applied the Kyosuke Takayama's
patch on my Redmine. But now it's gone due to the repository
support rewrite from JP.

Actions #11

Updated by Jean-Philippe Lang over 16 years ago

It's not implemented in current trunk (otherwise this ticket
would be closed).
I'll have a look at it.

Actions #12

Updated by Jean-Philippe Lang over 16 years ago

Hi,

This fix was finally committed in r814.
You can set the list of encodings in Admin -> Settings
(Repositories encodings).

Thanks.

Actions #13

Updated by Nikolay Solakov over 16 years ago

Thanks, again and again.

Nikolay

Actions

Also available in: Atom PDF