https://www.redmine.org/https://www.redmine.org/favicon.ico?16793021292016-06-13T13:48:47ZRedmineRedmine - Defect #23055: Error with Fetch commits with Mercurial repository when log has invalid charhttps://www.redmine.org/issues/23055?journal_id=714992016-06-13T13:48:47ZToshi MARUYAMA
<ul><li><strong>Status</strong> changed from <i>New</i> to <i>Needs feedback</i></li></ul><p>What output of scm_stderr_log_file?<br /><a class="source" href="https://www.redmine.org/projects/redmine/repository/svn/entry/tags/3.2.3/config/configuration.yml.example#L138">source:tags/3.2.3/config/configuration.yml.example#L138</a></p> Redmine - Defect #23055: Error with Fetch commits with Mercurial repository when log has invalid charhttps://www.redmine.org/issues/23055?journal_id=715082016-06-13T17:11:49Zmole mole
<ul></ul><p>I haven't any exit on scm_stderr_log_file.</p>
<p>But in production.log when do the first scan logged this:</p>
<pre>
Changeset Load (0.8ms) SELECT `changesets`.* FROM `changesets` WHERE `changesets`.`repository_id` = 11 AND `changesets`.`scmid` = '6e892cac598ab919f853ca54c7745e783b70dcb8' ORDER BY changesets.id DESC LIMIT 1
Changeset Exists (0.6ms) SELECT 1 AS one FROM `changesets` WHERE (`changesets`.`revision` = BINARY '10555' AND `changesets`.`id` != 37510 AND `changesets`.`repository_id` = 11) LIMIT 1
Changeset Exists (0.5ms) SELECT 1 AS one FROM `changesets` WHERE (`changesets`.`scmid` = BINARY '6e892cac598ab919f853ca54c7745e783b70dcb8' AND `changesets`.`id` != 37510 AND `changesets`.`repository_id` = 11) LIMIT 1
Changeset Exists (0.4ms) SELECT 1 AS one FROM `changesets` WHERE (`changesets`.`revision` = BINARY '10598' AND `changesets`.`repository_id` = 11) LIMIT 1
Changeset Exists (0.4ms) SELECT 1 AS one FROM `changesets` WHERE (`changesets`.`scmid` = BINARY '6289f36f5adf23615f6fc2f49abd5828beffb29b' AND `changesets`.`repository_id` = 11) LIMIT 1
SQL (0.5ms) INSERT INTO `changesets` (`repository_id`, `revision`, `scmid`, `committer`, `commit_date`, `committed_on`, `comments`, `user_id`) VALUES (11, '10598', '6289f36f5adf23615f6fc2f49abd5828beffb29b', 'pepe<eduardo@pepe.es>', '2011-07-15', '2011-07-15 07:42:42', 'TextStyle ahora senyala el estilo del caracter donde esta el cursor.', 6)
SQL (0.3ms) INSERT INTO `changeset_parents` (`parent_id`, `changeset_id`) VALUES (37510, 37553)
SQL (0.3ms) INSERT INTO `changes` (`changeset_id`, `action`, `path`) VALUES (37553, 'M', '/core/exe/edtext.cpp')
SQL (0.3ms) INSERT INTO `changes` (`changeset_id`, `action`, `path`) VALUES (37553, 'M', '/core/ipf/ctext.cpp')
SQL (0.3ms) INSERT INTO `changes` (`changeset_id`, `action`, `path`) VALUES (37553, 'M', '/core/ipf/ctext.h')
(34.2ms) COMMIT
(0.2ms) BEGIN
Changeset Load (0.7ms) SELECT `changesets`.* FROM `changesets` WHERE `changesets`.`repository_id` = 11 AND `changesets`.`scmid` = '6289f36f5adf23615f6fc2f49abd5828beffb29b' ORDER BY changesets.id DESC LIMIT 1
Changeset Exists (0.6ms) SELECT 1 AS one FROM `changesets` WHERE (`changesets`.`revision` = BINARY '10598' AND `changesets`.`id` != 37553 AND `changesets`.`repository_id` = 11) LIMIT 1
Changeset Exists (0.5ms) SELECT 1 AS one FROM `changesets` WHERE (`changesets`.`scmid` = BINARY '6289f36f5adf23615f6fc2f49abd5828beffb29b' AND `changesets`.`id` != 37553 AND `changesets`.`repository_id` = 11) LIMIT 1
Changeset Exists (0.4ms) SELECT 1 AS one FROM `changesets` WHERE (`changesets`.`revision` = BINARY '10599' AND `changesets`.`repository_id` = 11) LIMIT 1
Changeset Exists (0.4ms) SELECT 1 AS one FROM `changesets` WHERE (`changesets`.`scmid` = BINARY '2babe5c449c6274f052a8bda9637c247ea210e23' AND `changesets`.`repository_id` = 11) LIMIT 1
SQL (0.5ms) INSERT INTO `changesets` (`repository_id`, `revision`, `scmid`, `committer`, `commit_date`, `committed_on`, `comments`, `user_id`) VALUES (11, '10599', '2babe5c449c6274f052a8bda9637c247ea210e23', 'pepe<eduardo@pepe.es>', '2011-07-19', '2011-07-19 16:04:02', 'Soporte de estilos usando la seleccion de texto.', 6)
SQL (0.3ms) INSERT INTO `changeset_parents` (`parent_id`, `changeset_id`) VALUES (37553, 37554)
SQL (0.3ms) INSERT INTO `changes` (`changeset_id`, `action`, `path`) VALUES (37554, 'M', '/core/exe/edtext.cpp')
SQL (0.3ms) INSERT INTO `changes` (`changeset_id`, `action`, `path`) VALUES (37554, 'M', '/core/ipf/ctext.cpp')
SQL (0.4ms) INSERT INTO `changes` (`changeset_id`, `action`, `path`) VALUES (37554, 'M', '/core/ipf/ctext.h')
(26.2ms) COMMIT
Shelling out: 'hg' '-R' '/home/share/mirror/hg/all' '--encoding' 'utf-8' '--config' 'extensions.redminehelper=/usr/src/redmine/lib/redmine/scm/adapters/mercurial/redminehelper.py' '--config' 'diff.git=false' 'log' '--debug' '-C' '--style' '/usr/src/redmine/lib/redmine/scm/adapters/mercurial/hg-template-1.0.tmpl' '-r' '10600:10699'
Completed 500 Internal Server Error in 893414ms (ActiveRecord: 465576.7ms)
NoMethodError (undefined method `[]' for nil:NilClass):
lib/redmine/scm/adapters/mercurial_adapter.rb:194:in `each_revision'
app/models/repository/mercurial.rb:187:in `block in fetch_changesets'
app/models/repository/mercurial.rb:186:in `step'
app/models/repository/mercurial.rb:186:in `fetch_changesets'
app/controllers/repositories_controller.rb:114:in `show'
lib/redmine/sudo_mode.rb:63:in `sudo_mode'
</pre> Redmine - Defect #23055: Error with Fetch commits with Mercurial repository when log has invalid charhttps://www.redmine.org/issues/23055?journal_id=715192016-06-14T03:59:21ZToshi MARUYAMA
<ul></ul><p>mole mole wrote:</p>
<blockquote>
<p>I haven't any exit on scm_stderr_log_file.</p>
</blockquote>
<p>How about "#{Rails.env}.scm.stderr.log"?<br /><a class="source" href="https://www.redmine.org/projects/redmine/repository/svn/entry/tags/3.2.3/lib/redmine/scm/adapters/abstract_adapter.rb#L221">source:tags/3.2.3/lib/redmine/scm/adapters/abstract_adapter.rb#L221</a></p>
<blockquote>
<p>But in production.log when do the first scan logged this:<br />...<br />'hg' '-R' '/home/share/mirror/hg/all' '--encoding' 'utf-8' '--config' 'extensions.redminehelper=/usr/src/redmine/lib/redmine/scm/adapters/mercurial/redminehelper.py' '--config' 'diff.git=false' 'log' '--debug' '-C' '--style' '/usr/src/redmine/lib/redmine/scm/adapters/mercurial/hg-template-1.0.tmpl' '-r' '10600:10699'</p>
</blockquote>
<p>What output of above "hg" on command line?</p> Redmine - Defect #23055: Error with Fetch commits with Mercurial repository when log has invalid charhttps://www.redmine.org/issues/23055?journal_id=715392016-06-14T13:33:09Zmole mole
<ul><li><strong>File</strong> <a href="/attachments/16155">redmine_scm_stderr.log</a> <a class="icon-only icon-download" title="Download" href="/attachments/download/16155/redmine_scm_stderr.log">redmine_scm_stderr.log</a> added</li></ul><p>I haven't any exit on scm_stderr_log_file. -> I try to say: I haven't any error related with this issue. Only have this message: <br />not trusting file /home/share/mirror/hg/all/.hg/hgrc from untrusted user root, group www-data</p>
<p>It can be solve with adding in hgrc.<br /><pre>
[trusted]
users =
</pre></p>
<p>hg in command line:</p>
<pre>
Mercurial Distributed SCM
basic commands:
add add the specified files on the next commit
annotate show changeset information by line for each file
clone make a copy of an existing repository
commit commit the specified files or all outstanding changes
diff diff repository (or selected files)
export dump the header and diffs for one or more changesets
forget forget the specified files on the next commit
init create a new repository in the given directory
log show revision history of entire repository or files
merge merge working directory with another revision
pull pull changes from the specified source
push push changes to the specified destination
remove remove the specified files on the next commit
serve start stand-alone webserver
status show changed files in the working directory
summary summarize working directory state
update update working directory (or switch revisions)
use "hg help" for the full list of commands or "hg -v" for details
</pre> Redmine - Defect #23055: Error with Fetch commits with Mercurial repository when log has invalid charhttps://www.redmine.org/issues/23055?journal_id=715402016-06-14T14:00:42ZToshi MARUYAMA
<ul></ul><p>What is output of following command? <br />'hg' '-R' '/home/share/mirror/hg/all' '--encoding' 'utf-8' '--config' 'extensions.redminehelper=/usr/src/redmine/lib/redmine/scm/adapters/mercurial/redminehelper.py' '--config' 'diff.git=false' 'log' '--debug' '-C' '--style' '/usr/src/redmine/lib/redmine/scm/adapters/mercurial/hg-template-1.0.tmpl' '-r' '10600:10699'</p> Redmine - Defect #23055: Error with Fetch commits with Mercurial repository when log has invalid charhttps://www.redmine.org/issues/23055?journal_id=715442016-06-14T14:18:03Zmole mole
<ul><li><strong>File</strong> <a href="/attachments/16156">hgcommand.log</a> <a class="icon-only icon-download" title="Download" href="/attachments/download/16156/hgcommand.log">hgcommand.log</a> added</li></ul><p>The command 'hg' '-R' '/home/share/mirror/hg/all' '--encoding' 'utf-8' '--config' 'extensions.redminehelper=/usr/src/redmine/lib/redmine/scm/adapters/mercurial/redminehelper.py' '--config' 'diff.git=false' 'log' '--debug' '-C' '--style' '/usr/src/redmine/lib/redmine/scm/adapters/mercurial/hg-template-1.0.tmpl' '-r' '10600:10699'</p>
<p>Can be executed without errors. Attach output.</p> Redmine - Defect #23055: Error with Fetch commits with Mercurial repository when log has invalid charhttps://www.redmine.org/issues/23055?journal_id=715462016-06-14T14:48:28ZToshi MARUYAMA
<ul></ul><p>What is output?<br /><pre><code class="sql syntaxhl"><span class="k">select</span> <span class="n">revision</span> <span class="k">from</span> <span class="n">changesets</span> <span class="k">where</span> <span class="n">id</span> <span class="o">=</span>
<span class="p">(</span><span class="k">select</span> <span class="k">max</span><span class="p">(</span><span class="n">id</span><span class="p">)</span> <span class="k">from</span> <span class="n">changesets</span> <span class="k">where</span> <span class="n">repository_id</span> <span class="o">=</span> <span class="mi">11</span><span class="p">)</span>
</code></pre></p> Redmine - Defect #23055: Error with Fetch commits with Mercurial repository when log has invalid charhttps://www.redmine.org/issues/23055?journal_id=715472016-06-14T15:09:20Zmole mole
<ul></ul><p>revision <br />10599</p> Redmine - Defect #23055: Error with Fetch commits with Mercurial repository when log has invalid charhttps://www.redmine.org/issues/23055?journal_id=715482016-06-14T15:22:18ZToshi MARUYAMA
<ul></ul><p>Try reduce FETCH_AT_ONCE.<br /><a class="source" href="https://www.redmine.org/projects/redmine/repository/svn/entry/tags/3.1.3/app/models/repository/mercurial.rb#L30">source:tags/3.1.3/app/models/repository/mercurial.rb#L30</a></p>
<p>I think a revision from 10600 to 10699 has problem.</p> Redmine - Defect #23055: Error with Fetch commits with Mercurial repository when log has invalid charhttps://www.redmine.org/issues/23055?journal_id=715492016-06-14T15:31:55ZToshi MARUYAMA
<ul></ul>I think your revision has problem.
<ul>
<li>no date</li>
<li>no commit log</li>
</ul>
<p>If you find this revision, try this patch.<br /><pre><code class="diff syntaxhl"><span class="gh">diff --git a/lib/redmine/scm/adapters/mercurial_adapter.rb b/lib/redmine/scm/adapters/mercurial_adapter.rb
</span><span class="gd">--- a/lib/redmine/scm/adapters/mercurial_adapter.rb
</span><span class="gi">+++ b/lib/redmine/scm/adapters/mercurial_adapter.rb
</span><span class="p">@@ -212,8 +212,8 @@</span> module Redmine
yield Revision.new(:revision => le['revision'],
:scmid => le['node'],
:author => (le['author']['__content__'] rescue ''),
<span class="gd">- :time => Time.parse(le['date']['__content__']),
- :message => le['msg']['__content__'],
</span><span class="gi">+ :time => (Time.parse(le['date']['__content__']) rescue Time.at(0)),
+ :message => (le['msg']['__content__'] rescue ''),
</span> :paths => paths,
:parents => parents_ary)
end
</code></pre></p> Redmine - Defect #23055: Error with Fetch commits with Mercurial repository when log has invalid charhttps://www.redmine.org/issues/23055?journal_id=715512016-06-14T16:02:31Zmole mole
<ul><li><strong>File</strong> <a href="/attachments/16157">issue.jpg</a> <a class="icon-only icon-download" title="Download" href="/attachments/download/16157/issue.jpg">issue.jpg</a> added</li></ul><p>It didn't works. Same result.</p>
<p>Maybe a wrong code of EOL?. I see in sublime editor a DC3 code.</p> Redmine - Defect #23055: Error with Fetch commits with Mercurial repository when log has invalid charhttps://www.redmine.org/issues/23055?journal_id=715522016-06-14T16:28:00Zmole mole
<ul></ul><p>I thing that double <strong>!!</strong> can break it.</p>
<p>The message in mercurial is:</p>
<pre>
-dimensiones de los geosets de m3Font funcionando bien con shader.!!
</pre>
<p>and the output of script have a DC3 code when in original have a <strong>!!</strong>:</p>
<pre>
<msg>-dimensiones de los geosets de m3Font funcionando bien con shader.</msg>
</pre> Redmine - Defect #23055: Error with Fetch commits with Mercurial repository when log has invalid charhttps://www.redmine.org/issues/23055?journal_id=715562016-06-15T02:04:18ZToshi MARUYAMA
<ul></ul><p>Try upgrading Mercurial. Latest version is 3.8.3.</p> Redmine - Defect #23055: Error with Fetch commits with Mercurial repository when log has invalid charhttps://www.redmine.org/issues/23055?journal_id=715632016-06-15T09:01:33Zmole mole
<ul></ul><p>It didn't works. Same result.</p> Redmine - Defect #23055: Error with Fetch commits with Mercurial repository when log has invalid charhttps://www.redmine.org/issues/23055?journal_id=715642016-06-15T09:20:00ZToshi MARUYAMA
<ul></ul><p>Please report to Mercurial BTS.<br />Broken XML is Mercurial bug.<br /><a class="external" href="https://bz.mercurial-scm.org/">https://bz.mercurial-scm.org/</a></p> Redmine - Defect #23055: Error with Fetch commits with Mercurial repository when log has invalid charhttps://www.redmine.org/issues/23055?journal_id=715672016-06-15T09:46:55Zmole mole
<ul></ul><p>The character is ^S I try to reproduce in small repo</p> Redmine - Defect #23055: Error with Fetch commits with Mercurial repository when log has invalid charhttps://www.redmine.org/issues/23055?journal_id=715692016-06-15T09:58:36Zmole mole
<ul><li><strong>File</strong> <a href="/attachments/16158">all.tar.gz</a> <a class="icon-only icon-download" title="Download" href="/attachments/download/16158/all.tar.gz">all.tar.gz</a> added</li></ul><p>You can reproduce this issue with this repo.</p> Redmine - Defect #23055: Error with Fetch commits with Mercurial repository when log has invalid charhttps://www.redmine.org/issues/23055?journal_id=715702016-06-15T11:06:27ZToshi MARUYAMA
<ul></ul><p>It is obviously Mercurial bug.<br />Please report to Mercurial BTS.<br /><pre>
$ LANG=C /WEB-DOWN/hg-repo/hg-crew/hg log --version
Mercurial Distributed SCM (version 3.8.3)
(see https://mercurial-scm.org for more information)
Copyright (C) 2005-2016 Matt Mackall and others
This is free software; see the source for copying conditions. There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
$ /WEB-DOWN/hg-repo/hg-crew/hg log --debug --style /REDMINE/MY-REDMINE/test-trunk/lib/redmine/scm/adapters/mercurial/hg-template-1.0.tmpl > hg.xml
$ xmllint hg.xml
hg.xml:10: parser error : PCDATA invalid Char value 19
<msg>-Test</msg>
^
hg.xml:17: parser error : Premature end of data in tag log line 2
</pre></p> Redmine - Defect #23055: Error with Fetch commits with Mercurial repository when log has invalid charhttps://www.redmine.org/issues/23055?journal_id=715712016-06-15T11:16:29ZToshi MARUYAMA
<ul></ul><p>Sorry, we need add "</log>".</p>
<pre>
$ /WEB-DOWN/hg-repo/hg-crew/hg log --debug --style /REDMINE/MY-REDMINE/test-trunk/lib/redmine/scm/adapters/mercurial/hg-template-1.0.tmpl > hg.xml
$ echo "</log>" >> hg.xml
$ xmllint hg.xml
hg.xml:23: parser error : PCDATA invalid Char value 19
<msg>-Test</msg>
^
</pre> Redmine - Defect #23055: Error with Fetch commits with Mercurial repository when log has invalid charhttps://www.redmine.org/issues/23055?journal_id=715722016-06-15T11:28:30Zmole mole
<ul></ul><p>Using Mercurial xml template generate a correct xml</p>
<pre>
$ hg log --debug --style=xml > hg.xml
$ xmllint hg.xml
</pre>
<p>It runs correctly</p>
<p>PS: --style is a deprecated parameter in mercurial</p> Redmine - Defect #23055: Error with Fetch commits with Mercurial repository when log has invalid charhttps://www.redmine.org/issues/23055?journal_id=715732016-06-15T11:44:11Zmole mole
<ul></ul><p>I changed in hg-template-1.0.tmpl {desc|escape} by {desc|revscape} and xmllint runs correctly...</p> Redmine - Defect #23055: Error with Fetch commits with Mercurial repository when log has invalid charhttps://www.redmine.org/issues/23055?journal_id=715742016-06-15T11:53:28Zmole mole
<ul></ul><p>Sorry, its probably we must use xmlescape</p>
<p>{desc|xmlescape}</p>
<p>I see this in this file sample of mercurial</p>
<pre>
/usr/share/mercurial/templates/map-cmdline.xml
</pre> Redmine - Defect #23055: Error with Fetch commits with Mercurial repository when log has invalid charhttps://www.redmine.org/issues/23055?journal_id=715752016-06-15T12:08:59Zmole mole
<ul><li><strong>File</strong> <a href="/attachments/16159">hg-template-1.0.tmpl</a> <a class="icon-only icon-download" title="Download" href="/attachments/download/16159/hg-template-1.0.tmpl">hg-template-1.0.tmpl</a> added</li></ul><p>Attach hg-template-1.0.tmpl modified</p> Redmine - Defect #23055: Error with Fetch commits with Mercurial repository when log has invalid charhttps://www.redmine.org/issues/23055?journal_id=715762016-06-15T12:11:41Zmole mole
<ul><li><strong>File</strong> <a href="/attachments/16160">hg-template-1.0.tmpl</a> <a class="icon-only icon-download" title="Download" href="/attachments/download/16160/hg-template-1.0.tmpl">hg-template-1.0.tmpl</a> added</li></ul><p>Sorry, is this file</p> Redmine - Defect #23055: Error with Fetch commits with Mercurial repository when log has invalid charhttps://www.redmine.org/issues/23055?journal_id=715782016-06-15T12:39:54ZToshi MARUYAMA
<ul><li><strong>Subject</strong> changed from <i>Error with Fetch commits automatically with mercurial repository</i> to <i>Error with Fetch commits with Mercurial repository when log has invalid char</i></li><li><strong>Status</strong> changed from <i>Needs feedback</i> to <i>Confirmed</i></li><li><strong>Assignee</strong> set to <i>Toshi MARUYAMA</i></li></ul> Redmine - Defect #23055: Error with Fetch commits with Mercurial repository when log has invalid charhttps://www.redmine.org/issues/23055?journal_id=715792016-06-15T12:41:15Zmole mole
<ul></ul><p>Thanks for your help.</p> Redmine - Defect #23055: Error with Fetch commits with Mercurial repository when log has invalid charhttps://www.redmine.org/issues/23055?journal_id=747482016-11-28T12:58:01ZOscar Edvardsson
<ul></ul><p>I can also confirm this. Any workaround?</p>
<p>Environment:<br /> Redmine version 3.2.0.stable<br /> Ruby version 2.3.1-p112 (2016-04-26) [x86_64-linux-gnu]<br /> Rails version 4.2.5<br /> Environment production<br /> Database adapter PostgreSQL<br />SCM:<br /> Mercurial 3.7.3</p> Redmine - Defect #23055: Error with Fetch commits with Mercurial repository when log has invalid charhttps://www.redmine.org/issues/23055?journal_id=969522020-03-26T01:41:11ZToshi MARUYAMA
<ul><li><strong>Status</strong> changed from <i>Confirmed</i> to <i>Closed</i></li><li><strong>Target version</strong> set to <i>4.2.0</i></li><li><strong>Resolution</strong> set to <i>Fixed</i></li></ul><p>I choose 'urlescape' because 'paths' uses 'urlescape' and decodes it.<br /><a class="source" href="https://www.redmine.org/projects/redmine/repository/svn/revisions/19606/entry/trunk/lib/redmine/scm/adapters/mercurial_adapter.rb#L204">source:trunk/lib/redmine/scm/adapters/mercurial_adapter.rb@19606#L204</a></p>