Defect #10887
openCan't fetch Mercurial changesets from repository
0%
Description
I get this backtrace when trying to view a repository:
NoMethodError (undefined method `[]' for nil:NilClass): lib/redmine/scm/adapters/mercurial_adapter.rb:206:in `each_revision' app/models/repository/mercurial.rb:141:in `fetch_changesets' app/models/repository/mercurial.rb:140:in `step' app/models/repository/mercurial.rb:140:in `fetch_changesets' app/controllers/repositories_controller.rb:91:in `show' /usr/lib/ruby/1.8/rack/adapter/rails.rb:74:in `call' /usr/lib/ruby/1.8/thin/connection.rb:76:in `pre_process' /usr/lib/ruby/1.8/thin/connection.rb:74:in `catch' /usr/lib/ruby/1.8/thin/connection.rb:74:in `pre_process' /usr/lib/ruby/1.8/thin/connection.rb:57:in `process' /usr/lib/ruby/1.8/thin/connection.rb:42:in `receive_data' /usr/lib/ruby/1.8/eventmachine.rb:257:in `run_machine' /usr/lib/ruby/1.8/eventmachine.rb:257:in `run' /usr/lib/ruby/1.8/thin/backends/base.rb:57:in `start' /usr/lib/ruby/1.8/thin/server.rb:156:in `start' /usr/lib/ruby/1.8/thin/controllers/controller.rb:80:in `start' /usr/lib/ruby/1.8/thin/runner.rb:174:in `send' /usr/lib/ruby/1.8/thin/runner.rb:174:in `run_command' /usr/lib/ruby/1.8/thin/runner.rb:140:in `run!' /usr/bin/thin1.8:6
The repository in question has over 3000 changesets.
Output from script/about:
About your application's environment Ruby version 1.8.7 (x86_64-linux) RubyGems version 1.3.7 Rack version 1.1.3 Rails version 2.3.14 Active Record version 2.3.14 Active Resource version 2.3.14 Action Mailer version 2.3.14 Active Support version 2.3.14 Application root /var/lib/redmine Environment production Database adapter mysql Database schema version 20120301153455
Running Redmine 1.4-stable -> thin -> nginx.
Updated by Vasiliy Marchuk over 12 years ago
I get this too:
NoMethodError (undefined method `[]' for nil:NilClass): lib/redmine/scm/adapters/mercurial_adapter.rb:206:in `each_revision' app/models/repository/mercurial.rb:141:in `fetch_changesets' app/models/repository/mercurial.rb:140:in `step' app/models/repository/mercurial.rb:140:in `fetch_changesets' app/controllers/repositories_controller.rb:114:in `show'
Repository only has 14 commits:
@ 14[tip] 2272237fef14 2012-03-11 20:34 +0200 styx | Fixed sending signature to Qbix main site from reports script | o 13 3f3b5c39359b 2012-02-17 02:36 +0200 styx | Reports cron script | o 12 f79cf76626fe 2011-07-30 12:01 +0300 styx | disable sql-dump into errors.log | o 11 3297edd6c8dc 2011-07-30 11:58 +0300 styx | little fix | o 10 fdeb01db8881 2011-07-30 11:48 +0300 styx | support for 1.3 app version, added BundleVersion and SystemVersion into Pings' table | o 9 526ab3d6615f 2011-04-14 01:37 -0400 gregory | redirecting to qbix.com/calendar now | o 8 a11db5f4d074 2011-04-13 04:38 -0400 gregory | chmod 770 | o 7 aecda1a96b7a 2011-03-10 18:06 +0200 styx | Support for ping's messages | o 6:5,4 4e597fb30296 2011-03-07 20:02 +0200 styx |\ Merge with main | | | o 5:3 556e586c1c6d 2011-03-06 17:55 +0200 styx | | Web side for Calendar pings | | o | 4 e879e2095c7d 2011-03-06 13:55 -0500 greg |/ testing if we can commit | o 3 8f001bc4f6d9 2011-03-03 06:07 -0500 gregory | changed route | o 2 d46010c6b2ec 2011-03-03 06:06 -0500 gregory | renamed GroupsApi to CalendarApi | o 1 f893d8a34085 2011-03-03 06:06 -0500 gregory | added support for pings | o 0 dddfadc3157b 2011-03-03 05:53 -0500 gregory calendar app
Running Redmine 2.0.0.stable -> thin -> nginx
script/about:
Environment: Redmine version 2.0.0.stable Ruby version 1.8.7 (i386-linux) Rails version 3.2.3 Environment production Database adapter MySQL Redmine plugins: no plugin installed
Mercurial 2.0
Updated by Anonymous about 12 years ago
I also get this issue, with a repository with almost 18000 commits. On the web interface, it shows all commits up to and including 14199.
Environment: Redmine version 2.0.3.stable Ruby version 1.9.3 (x86_64-linux) Rails version 3.2.6 Environment production Database adapter Mysql2 Redmine plugins: redmine_auto_percent 0.0.1 redmine_favourite_projects 0.5 redmine_latex_mathjax 0.1.0 redmine_openid_selector 0.0.1 redmine_recaptcha 0.1.0
Mercurial 2.2.2
Updated by Anonymous about 12 years ago
Actually, I should clarify: I get an issue that looks very similar, but I am not 100% sure if it is the same. Namely, when I run ./script/rails runner "Repository.fetch_changesets" -e production
, I get the following error (with some paths slightly "anonymized"):
/PATH-USER/gems/railties-3.2.6/lib/rails/commands/runner.rb:53:in `eval': undefined method `[]' for nil:NilClass (NoMethodError) from /PATH-WWW/app/models/repository/mercurial.rb:141:in `block in fetch_changesets' from /PATH-WWW/app/models/repository/mercurial.rb:140:in `step' from /PATH-WWW/app/models/repository/mercurial.rb:140:in `fetch_changesets' from /PATH-WWW/app/models/repository.rb:306:in `block (2 levels) in fetch_changesets' from /PATH-USER/gems/activerecord-3.2.6/lib/active_record/associations/collection_proxy.rb:89:in `each' from /PATH-USER/gems/activerecord-3.2.6/lib/active_record/associations/collection_proxy.rb:89:in `method_missing' from /PATH-WWW/app/models/repository.rb:304:in `block in fetch_changesets' from /PATH-WWW/app/models/repository.rb:303:in `each' from /PATH-WWW/app/models/repository.rb:303:in `fetch_changesets' from (eval):1:in `<top (required)>' from /PATH-USER/gems/railties-3.2.6/lib/rails/commands/runner.rb:53:in `eval' from /PATH-USER/gems/railties-3.2.6/lib/rails/commands/runner.rb:53:in `<top (required)>' from /PATH-USER/gems/railties-3.2.6/lib/rails/commands.rb:64:in `require' from /PATH-USER/gems/railties-3.2.6/lib/rails/commands.rb:64:in `<top (required)>' from ./script/rails:6:in `require' from ./script/rails:6:in `<main>'
Updated by Vadim Kotov about 12 years ago
Got the same error with Redmine 2.0.4 and Mercurial 2.3.1. Deleting repo in the project's settings and adding it again did not help. Any ideas why is this happening? SCM bindings are completely unusable due to this error.
Updated by Vadim Kotov about 12 years ago
Ok, I've done a small research and got this thing fixed. In my case there were additional lines from Mercurial in the hg log output, reporting that repo's hgrc file is not trusting (due to permission issue). Thats why Redmine failed to parse that output and changeset collection was nil.
I've added hgrc owner to the trusted section, and changesets were parsed successfully (refer to Mercurial Trust). Also, this issue affects 2.x Redmine branch. I guess there should be a fallback if there are some wrong lines in the log.
Hope this info could help someone!
Updated by Anonymous over 11 years ago
While I am also seeing these messages about hg not trusting some hgrc file, this proved to be a red herring in my case -- I successfully turned off the warning, but it didn't help.
But I finally figured out that the problem was a specific commit that redmine choked on. First, to figure out what commit was causing the problem, I modified app/models/repository/mercurial.rb by changing FETCH_AT_ONCE from 100 to 1 -- this is slower, but this way, I was able to pinpoint the precise commit redmine was choking on.
Then I had a closer look. At first I was stumped, but then I changed the XML parser being used to libxml2, which seems to be more strict / give better diagnostics. This required installing the libxml-ruby gem, and then editing lib/redmine/scm/adapters/abstract_adapter.rb where I inserted the line
ActiveSupport::XmlMini.backend = 'LibXML'
right before
ActiveSupport::XmlMini.parse(xml)
After that, I triggered the repository import from the command line, as described in the Redmine user guide. I.e. via the command
./script/rails runner "Repository.fetch_changesets" -e production
This time it died with the message
Fatal error: PCDATA invalid Char value 24 at :10.
Value 24 is a non-printing ASCII character, which somehow snuck into the commit message of the offending commit. (I checked this by piping the output of "hg log" for that commit into a file and then inspecting this with a hex editor). In my case, the one-time solution thus was the following: I edited lib/redmine/scm/adapters/mercurial_adapter.rb and inserted
output.delete!("\x18")
right before this (which is at line 201 in Redmine 2.2.3; beware, there are more parse_xml call in that file):
begin # Mercurial < 1.5 does not support footer template for '</log>' parse_xml("#{output}</log>")['log'] rescue end
In words: I told Redmine to throw away any occurrences of the bad character.
My guess is that for other people, the problem might also be caused by problematic characters in the log message. It might be a different character, though, but I hope that my instructions will at least help some people with this problem.
Of course it would be nice if there was a "proper" fix for this. I imagine this could be similar to what I did hear, but should probably filter out more bad characters... or perhaps hg or the Redmine mercurial imported could be modified to generate XML using CDATA, instead of relying on the data to be PCDATA...
Updated by Ian German Mesner over 11 years ago
I can report this is still occurring due to certain characters, and I used Max Horn's solution.
I am unfamiliar with the redmine codebase and ruby, but I can see a bug with the xml parsing in mercurial_adapter.rb. Basically, any exception thrown by parse_xml is ignored, with nothing logged. It seems to me that an empty object should be returned (or a check for nil afterwards), and the offending scm log entry be logged by redmine. That way, the entire repository fetch won't be in error if a single scm log entry is corrupt (in my case an accidental DLE character was inserted). In addition, it would be nice to strip offending characters out of the log message.
195 log = hg(*hg_args) do |io| 196 output = io.read 197 if output.respond_to?(:force_encoding) 198 output.force_encoding('UTF-8') 199 end 200 begin 201 # Mercurial < 1.5 does not support footer template for '</log>' 202 parse_xml("#{output}</log>")['log'] 203 rescue 204 end 205 end
Updated by Marc Schlaich about 11 years ago
Seeing a similar issue
hg: error during getting info: undefined method `[]' for nil:NilClass Completed 500 Internal Server Error in 310ms NoMethodError (undefined method `[]' for nil:NilClass): lib/redmine/scm/adapters/mercurial_adapter.rb:161:in `entries' app/models/repository.rb:192:in `entries' app/controllers/repositories_controller.rb:116:in `show'
Updated by Egor Dolgalev almost 11 years ago
I have the same issue like Marc Schlaich
hg: error during getting info: undefined method `[]' for nil:NilClass Completed 500 Internal Server Error in 310ms NoMethodError (undefined method `[]' for nil:NilClass): lib/redmine/scm/adapters/mercurial_adapter.rb:161:in `entries' app/models/repository.rb:192:in `entries' app/controllers/repositories_controller.rb:116:in `show'
I choose debug log level and get:
Shelling out: "hg" "-R" "d:\tmp\1\test-branch\" "--encoding" "utf-8" "--config" "extensions.redminehelper=e:/redmine-2.4.1/lib/redmine/scm/adapters/mercurial/redminehelper.py" "--config" "diff.git=false" "rhmanifest" "-r" "tip" "" Completed 500 Internal Server Error in 397.0ms NoMethodError (undefined method `[]' for nil:NilClass): lib/redmine/scm/adapters/mercurial_adapter.rb:161:in `entries' app/models/repository.rb:192:in `entries' app/controllers/repositories_controller.rb:116:in `show'
Then I tried to run above command manually and get next log:
<?xml version="1.0"?> <rhmanifest> <repository root="d%3A%5Ctmp%5C1%5Ctest-branch"> <manifest revision="23" path=""> <file name=".hgtags" revision="22" node="dbecb64f3572" time="1379692530" size="224"/> <file name="file.txt" revision="20" node="d6dc83e4d9a8" time="1379692454" size="101"/> </manifest> </repository> </rhmanifest>
Updated by Jérôme Mestres about 7 years ago
Late response, but if I can help...
Same issue for me :
NoMethodError (undefined method `[]' for nil:NilClass): lib/redmine/scm/adapters/mercurial_adapter.rb:161:in `entries' app/models/repository.rb:192:in `entries' app/controllers/repositories_controller.rb:116:in `show'
Watch in "lib/redmine/scm/adapters/mercurial_adapter.rb:194" and I tried to execute manually the command at the top of methode :
hg log --debug -C --style /redmine/lib/redmine/scm/adapters/mercurial/hg-template-1.0.tmpl /repo/my-repo > output.log
The file is expect to be XML, but on first line (before <xml>) I have this :
ignoring untrusted configuration option hooks.changegroup [...]
So redmine, can not parse the output as XML.
Change my hgrc file to disable the hook, and It's work !!
I this can help...