Redmine

We noticed that the changesets were no longer showing up in our issues, and the redmine log showed the following output/error:

Started POST "/applsoft/github_hook?project_id=myprojname&repository_id=myreponame" for xxx.yyy.33.72 at Fri Dec 11 18:06:21 +1100 2015
Processing by GithubHookController#index as */*
  Parameters: {"project_id"=>"myprojname", "repository_id"=>"myreponame"}
  Current user: anonymous
Completed 500 Internal Server Error in 594ms

ArgumentError (comparison of String with 7 failed):
  app/models/repository/git.rb:142:in `sort'
  app/models/repository/git.rb:142:in `fetch_changesets'

We are running an older redmine 2.3.3.stable, however there were no changes to fetch_changesets between our version and git.rb on trunk.

I don't know ruby, but added some logger.info into fetch_changesets, and looks like prev_db_heads data is inconsistent/corrupt.

 ... "937b8ccd5d09c7a259541bd76e2ee593e7556462", "696fe403e863e6046f6cd00416f055d6bb02075b", 
"7d82fc17fc01f1e6a8babd5b7f89f3110e0df2ed", 7]

That's how it came from the database. Looks like we may have exceeded the limit for the extra_info text field in the repositories table with around 67063 bytes, apparently there may be a 65535 byte/char limit. (We use a lot of branches and pull requests for this repo)

So, now I've tried 'trimming' the data in that field, which avoids the "ArgumentError (comparison of String with 7 failed)" error, but now I get another error, Broken pipe:

Started POST "/applsoft/github_hook?project_id=myproject&repository_id=myrepo" for xxx.yyy.33.72 at Fri Dec 11 19:22:33 +1100 2015
Processing by GithubHookController#index as */*
  Parameters: {"project_id"=>"myproject", "repository_id"=>"myrepo"}
  Current user: anonymous
SCM command failed, make sure that your SCM command (e.g. svn) is in PATH (/usr/local/lib/ruby/gems/1.8/bin:/home/redmine/bin:/usr/local/bin:/usr/bin:/bin)
You can configure your scm commands in config/configuration.yml.
'git' '--git-dir' '/home/redmine/repositories/myrepo.git' '-c' 'core.quotepath=false' '-c' 'log.decorate=no' 'log' '--no-color'
 '--encoding=UTF-8' '--raw' '--date=iso' '--pretty=fuller' '--parents' '--stdin' '--reverse'
with: Broken pipe
Completed 500 Internal Server Error in 523ms

Redmine::Scm::Adapters::CommandFailed (Broken pipe):
  lib/redmine/scm/adapters/abstract_adapter.rb:276:in `shellout'
  lib/redmine/scm/adapters/abstract_adapter.rb:214:in `shellout'
  lib/redmine/scm/adapters/git_adapter.rb:402:in `git_cmd'
  lib/redmine/scm/adapters/git_adapter.rb:218:in `revisions'
  app/models/repository/git.rb:164:in `save_revisions'
  app/models/repository/git.rb:154:in `fetch_changesets'

And yes, git is in the path. Running the hook on another project with another repo completes successfully.

So I had a quick look at the data in this extra_info, I assume they are the last known hashes of branches. (Oh, and where does redmine store the list of branches?)

Some of these hashes were not valid, about 40 out of the 1500 in the list. Checking git log -1 gave 'bad object' and git cat-file returned fatal instead of verifying they are commits.

After manually removing these from the field in the database, running the hook post results in a git error:

Started POST "/applsoft/github_hook?project_id=myproject&repository_id=myrepo" for xxx.yyy.33.72 at Fri Dec 11 21:00:19 +1100 2015
Processing by GithubHookController#index as */*
  Parameters: {"project_id"=>"myproject", "repository_id"=>"myrepo"}
  Current user: anonymous
git log error: git exited with non-zero status: 128
  Rendered text template (0.0ms)
Completed 200 OK in 9164ms (Views: 0.8ms | ActiveRecord: 1.7ms)

What should I try next?
What would happen if deleted everything in the extra_info field for this repo?

If I can somehow force a re-sync, either by deleting that or some other method, would that result in duplicate commits showing up in the 'Associated revisions' section of an issue?