Redmine

No feedback. Reopen if needed.

Today i've updated (to r1923), and it works a lot better. I probably had a too old version. Keep up the good work.

This happens for me too on a Git repository. I just added it to our redmine today, loaded the change sets on the command line (6900 of them). Now whenever I click on Repository the site hangs and the browser times out. In debug mode I see that it takes redmine 248 seconds to complete - each and every time.

The delay seems to come from redmine doing a "git --git-dir '/mnt/data-store/redmine/git-repositories/fishpond/' log --date=iso --pretty=fuller 1 master - 'faq.php'" call on all 300 top level files that the "git --git-dir '/mnt/data-store/redmine/git-repositories/fishpond/' ls-tree -l 'HEAD:'" lists.

I upgraded to the latest in trunk (r2042) to see if there was a fix but it didn't help.

I'm pretty sure it is the number of files in the directory that make the difference rather than the revision count as David Kowis's example has what looks to be 100 files in his top-level directory and it is about 1/3 as fast. Redmine's own svn repo trunk/app/views with 37 folders takes a few seconds to load.

Here is a summary of the debug output for our repo:

Processing RepositoriesController#show (for 127.0.0.12 at 2008-11-18 01:01:26) [GET]
  Session ID: a6af16ec88ef0b45e2e5c6a53188149a
  Parameters: {"action"=>"show", "id"=>"bgz", "controller"=>"repositories"}
  SQL (0.000101)   SELECT max(`settings`.updated_on) AS max_updated_on FROM `settings`
  User Load (0.000057)   SELECT * FROM `users` WHERE (`users`.`id` = 3) AND (status = 1)
  Project Load (0.000061)   SELECT * FROM `projects` WHERE (`projects`.`identifier` = 'bgz') LIMIT 1
  Repository Load (0.000045)   SELECT * FROM `repositories` WHERE (`repositories`.project_id = 1) LIMIT 1
  EnabledModule Load (0.000065)   SELECT * FROM `enabled_modules` WHERE (`enabled_modules`.project_id = 1)
Shelling out: git --git-dir '/mnt/data-store/redmine/git-repositories/fishpond/' log --raw --date=iso --pretty=fuller -n 1
  Changeset Load (0.000130)   SELECT * FROM `changesets` WHERE (`changesets`.repository_id = 4) ORDER BY changesets.committed_on DESC, changesets.id DESC LIMIT 1
  Changeset Load (0.000062)   SELECT * FROM `changesets` WHERE (`changesets`.`scmid` = '268b9f207d6dbb880f353b1e1a934e926140dc5e') AND (`changesets`.repository_id = 4) ORDER BY changesets.committed_on DESC, changesets.id DESC LIMIT 1
Shelling out: git --git-dir '/mnt/data-store/redmine/git-repositories/fishpond/' ls-tree -l 'HEAD:'
Shelling out: git --git-dir '/mnt/data-store/redmine/git-repositories/fishpond/' branch
Shelling out: git --git-dir '/mnt/data-store/redmine/git-repositories/fishpond/' log --date=iso --pretty=fuller -1 master -- '.cvsignore'
Shelling out: git --git-dir '/mnt/data-store/redmine/git-repositories/fishpond/' branch
Shelling out: git --git-dir '/mnt/data-store/redmine/git-repositories/fishpond/' log --date=iso --pretty=fuller -1 master -- '.gitignore'
--the branch and log commands repeat for the 300 files--
  Changeset Load (0.000159)   SELECT * FROM `changesets` WHERE (`changesets`.repository_id = 4) ORDER BY committed_on DESC, changesets.committed_on DESC, changesets.id DESC LIMIT 10
Rendering template within layouts/base
Rendering repositories/show
  Changeset Load (0.000123)   SELECT * FROM `changesets` WHERE (`changesets`.`revision` = 'f5f46f9f01c29a3414260e83fa483d75d45813bd') AND (`changesets`.repository_id = 4) ORDER BY changesets.committed_on DESC, changesets.id DESC LIMIT 1
  User Load (0.000059)   SELECT * FROM `users` WHERE (`users`.`id` = 9)
  Changeset Load (0.000082)   SELECT * FROM `changesets` WHERE (`changesets`.`revision` = '59dd7ab40a541ace8256ff5f1af9d50d39b1aed6') AND (`changesets`.repository_id = 4) ORDER BY changesets.committed_on DESC, changesets.id DESC LIMIT 1
  User Load (0.000058)   SELECT * FROM `users` WHERE (`users`.`id` = 51)
  Changeset Load (0.000104)   SELECT * FROM `changesets` WHERE (`changesets`.`revision` = '0c8ba9ce1711e9887786ec9b9d2b017659a24ab3') AND (`changesets`.repository_id = 4) ORDER BY changesets.committed_on DESC, changesets.id DESC LIMIT 1
  CACHE (0.000000)   SELECT * FROM `users` WHERE (`users`.`id` = 51)
--sql like this occurs 300 or so times--
  CACHE (0.000000)   SELECT * FROM `users` WHERE (`users`.`id` = 4)
Rendered repositories/_dir_list_content (2.26420)
Rendered repositories/_dir_list (2.26477)
  UserPreference Load (0.000118)   SELECT * FROM `user_preferences` WHERE (`user_preferences`.user_id = 3) LIMIT 1
  CACHE (0.000000)   SELECT * FROM `users` WHERE (`users`.`id` = 10)
  CACHE (0.000000)   SELECT * FROM `users` WHERE (`users`.`id` = 4)
  CACHE (0.000000)   SELECT * FROM `users` WHERE (`users`.`id` = 4)
  CACHE (0.000000)   SELECT * FROM `users` WHERE (`users`.`id` = 4)
  CACHE (0.000000)   SELECT * FROM `users` WHERE (`users`.`id` = 42)
  CACHE (0.000000)   SELECT * FROM `users` WHERE (`users`.`id` = 42)
  CACHE (0.000000)   SELECT * FROM `users` WHERE (`users`.`id` = 42)
  CACHE (0.000000)   SELECT * FROM `users` WHERE (`users`.`id` = 4)
  CACHE (0.000000)   SELECT * FROM `users` WHERE (`users`.`id` = 42)
  CACHE (0.000000)   SELECT * FROM `users` WHERE (`users`.`id` = 10)
Rendered repositories/_revisions (0.04287)
  Token Load (0.000123)   SELECT * FROM `tokens` WHERE (`tokens`.user_id = 3 AND (action='feeds')) LIMIT 1
  SQL (0.000117)   SELECT count(DISTINCT `members`.id) AS count_all FROM `members` LEFT OUTER JOIN `projects` ON `projects`.id = `members`.project_id LEFT OUTER JOIN `roles` ON `roles`.id = `members`.role_id WHERE (`members`.user_id = 3 AND (projects.status=1))
  Project Load Including Associations (0.000110)   SELECT `projects`.`id` AS t0_r0, `projects`.`name` AS t0_r1, `projects`.`description` AS t0_r2, `projects`.`homepage` AS t0_r3, `projects`.`is_public` AS t0_r4, `projects`.`parent_id` AS t0_r5, `projects`.`projects_count` AS t0_r6, `projects`.`created_on` AS t0_r7, `projects`.`updated_on` AS t0_r8, `projects`.`identifier` AS t0_r9, `projects`.`status` AS t0_r10, `parents_projects`.`id` AS t1_r0, `parents_projects`.`name` AS t1_r1, `parents_projects`.`description` AS t1_r2, `parents_projects`.`homepage` AS t1_r3, `parents_projects`.`is_public` AS t1_r4, `parents_projects`.`parent_id` AS t1_r5, `parents_projects`.`projects_count` AS t1_r6, `parents_projects`.`created_on` AS t1_r7, `parents_projects`.`updated_on` AS t1_r8, `parents_projects`.`identifier` AS t1_r9, `parents_projects`.`status` AS t1_r10 FROM `projects` LEFT OUTER JOIN `projects` parents_projects ON `parents_projects`.id = `projects`.parent_id INNER JOIN members ON projects.id = members.project_id WHERE ((`members`.user_id = 3) AND ((projects.status=1)))
Rendered layouts/_project_selector (0.00469)
  SQL (0.000106)   SELECT count(*) AS count_all FROM `versions` WHERE (`versions`.project_id = 1)
  Wiki Load (0.000048)   SELECT * FROM `wikis` WHERE (`wikis`.project_id = 1) LIMIT 1
  SQL (0.000042)   SELECT count(*) AS count_all FROM `boards` WHERE (`boards`.project_id = 1)
Completed in 224.48472 (0 reqs/sec) | Rendering: 2.71917 (1%) | DB: 0.01270 (0%) | 200 OK [http://redmine.fishpond.co.nz/repositories/show/bgz]

Thanks for the info.
Here is the patch. It's not thoroughly tested but it seems to work.
First access may be slow (could be improved by adding an index on the "path" column of the table "changes"). But once there is some cache at db level, it's pretty fast.

Note that it doesn't give the exact same results as using git log on the file (eg. a git commit do not report to change a file when running git log <commit> but some files are reported to be modified by the commit when running git log -- <path>). Maybe there's a reason for this, but I'm really not mastering git.

Okay, with this slight variation on the patch (attached) (which relies on changeset_id's being monotonically increasing) and

create index changes_path_id on changes(path,changeset_id)

I get really quite snappy performance.

Original Mercurial fetching revisions performance problem is fixed by r4611.
I close this issue.

Git issues are followings.

• git fetching revisions performance problem (#6013)
• git browsing directory performance problem (#7047)