Defect #7047: Git adapter very slow when a commit modifies a lot of files - Redmine

Actions

Copy link

Defect #7047

open

Git adapter very slow when a commit modifies a lot of files

Added by Jean-Baptiste Barth about 14 years ago. Updated almost 12 years ago.

Status:

New

Priority:

Low

Assignee:

Category:

SCM

Target version:

Start date:

2010-12-04

Due date:

% Done:

Estimated time:

Resolution:

Affected version:

Description

I have a Git repo with thousands of files modified in some commits. Repository index action is very slow (>20s).

At first sight, it seems it's due to Redmine::Scm::Adapters::GitAdapter#revisions uses --raw option, which is not needed in all cases, at least not needed in Repository::Git#latest_changesets. It results in thousands of useless lines parsed after the shell-out.

I'll have a look at that. I admit it's not a common use case, hence the priority.

Files

Download all files

false_flag_report_last_commit.png (96 KB) false_flag_report_last_commit.png		Toshi MARUYAMA, 2011-03-03 09:25
true_flag_report_last_commit.png (145 KB) true_flag_report_last_commit.png		Toshi MARUYAMA, 2011-03-03 09:25

Related issues

Related to Redmine - Defect #1435: slow display of 'repository' tab for huge repository (pagination problem?)

Closed

2008-06-12

Actions

Related to Redmine - Defect #5096: Redmine hangs up while browsing Git repository

Closed

Toshi MARUYAMA

2010-03-16

Actions

Related to Redmine - Defect #4773: Redmine+Git+PostgresSQL 8.4 fails with linux kernel tree (encoding)

Closed

Jean-Philippe Lang

2010-02-09

Actions

Related to Redmine - Feature #6092: Truncate Git revision labels in Activity page/feed and allow configurable length

Closed

Toshi MARUYAMA

2010-08-09

Actions

Related to Redmine - Defect #5357: Git: SCM revisions ordered by date/time (should be reverse commit order)

New

Toshi MARUYAMA

2010-04-20

Actions

Related to Redmine - Defect #6013: git tab,browsing, very slow -- even after first time

Closed

2010-08-02

Actions

Related to Redmine - Feature #8365: Git: per project setting to report last commit or not in repository tree

Closed

Toshi MARUYAMA

2011-05-13

Actions

Copy link

Updated by Toshi MARUYAMA about 14 years ago

Please see http://www.redmine.org/issues/4773#note-14 and try patch http://www.redmine.org/attachments/3272/git-fast-browse.patch

Actions

Copy link

Updated by Toshi MARUYAMA about 14 years ago

Please see following links.

Actions

Copy link

Updated by Toshi MARUYAMA almost 14 years ago

$ time wget https://www.chiliproject.org/projects/chiliproject/repository
--2011-02-08 10:33:38--  https://www.chiliproject.org/projects/chiliproject/repository
Resolving www.chiliproject.org... 184.171.175.157
Connecting to www.chiliproject.org|184.171.175.157|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 30894 (30K) [text/html]
Saving to: `repository'

100%[================================================================================================>] 30,894      --.-K/s   in 0.1s    

2011-02-08 10:33:41 (213 KB/s) - `repository' saved [30894/30894]

real    0m3.378s
user    0m0.063s
sys     0m0.023s

$ time wget https://www.chiliproject.org/projects/chiliproject/repository/revisions/master/show/db/migrate
--2011-02-08 10:33:54--  https://www.chiliproject.org/projects/chiliproject/repository/revisions/master/show/db/migrate
Resolving www.chiliproject.org... 184.171.175.157
Connecting to www.chiliproject.org|184.171.175.157|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 139043 (136K) [text/html]
Saving to: `migrate'

100%[================================================================================================>] 139,043      216K/s   in 0.6s    

2011-02-08 10:34:20 (216 KB/s) - `migrate' saved [139043/139043]

real    0m26.036s
user    0m0.068s
sys     0m0.024s

Actions

Copy link

Updated by Etienne Massip almost 14 years ago

Assignee changed from Jean-Baptiste Barth to Toshi MARUYAMA

Actions

Copy link

Updated by Toshi MARUYAMA almost 14 years ago

The issue cannot be solved to no use of wrap git command such as Yuya's Mercurial extension.

source:trunk/lib/redmine/scm/adapters/mercurial/redminehelper.py

Actions

Copy link

Updated by Toshi MARUYAMA almost 14 years ago

File false_flag_report_last_commit.png false_flag_report_last_commit.png added
File true_flag_report_last_commit.png true_flag_report_last_commit.png added
Assignee deleted (~~Toshi MARUYAMA~~)

I added flag to switch showing last commit at source:trunk/lib/redmine/scm/adapters/git_adapter.rb@4992#L25.

true¶

$ time wget http://localhost:3000/projects/redmine-git-00/repository/revisions/master/show/db/migrate
--2011-02-24 21:17:23--  http://localhost:3000/projects/redmine-git-00/repository/revisions/master/show/db/migrate
Resolving localhost... 127.0.0.1
Connecting to localhost|127.0.0.1|:3000... connected.
HTTP request sent, awaiting response... 200 OK
Length: 139135 (136K) [text/html]
Saving to: `migrate'

100%[===================================================================================>] 139,135     --.-K/s   in 0.009s  

2011-02-24 21:18:11 (15.5 MB/s) - `migrate' saved [139135/139135]

real    0m47.799s
user    0m0.004s
sys    0m0.013s

false¶

$ time wget http://localhost:3000/projects/redmine-git-00/repository/revisions/master/show/db/migrate
--2011-02-24 21:20:43--  http://localhost:3000/projects/redmine-git-00/repository/revisions/master/show/db/migrate
Resolving localhost... 127.0.0.1
Connecting to localhost|127.0.0.1|:3000... connected.
HTTP request sent, awaiting response... 200 OK
Length: 100345 (98K) [text/html]
Saving to: `migrate'

100%[===================================================================================>] 100,345     --.-K/s   in 0.007s  

2011-02-24 21:20:45 (14.7 MB/s) - `migrate' saved [100345/100345]

real    0m2.076s
user    0m0.005s
sys    0m0.007s

I don't have a plan to refactor git adapter.

Actions

Copy link