Redmine slow Repository view, everything else fast :/

Added by Roy Sindre Norangshol over 14 years ago

First of all, the redmine deployment is quick and fast until we try to visit the Repository view. We're using Redmine.pm (perl) module for subversion authentication towards redmine and webdav and enabled it's cache too.

We notice the requests take up to 7 seconds due to the webdav svn requests. But, we have made sure to run the cronjob and fetch changesets etc every 30min, so Redmine shouldn't be needing to process those requests when browsing the subversion repository. We have restarted apache to make sure phassenger is "rebooting" the redmine application incase settings didn't take effect. But for some odd reason, it still continune to pull svn requests even tho we have explicitly set it not to. (by unchecking to fetch subversion changesets under Administration and Settings.)

From apache2 log:

server.ip - - [02/May/2010:19:57:31 +0200] "OPTIONS /svn/dotkom HTTP/1.1" 200 - "-" "SVN/1.5.1 (r32289) neon/0.28.2" 
server.ip - - [02/May/2010:19:57:31 +0200] "PROPFIND /svn/dotkom HTTP/1.1" 207 340 "-" "SVN/1.5.1 (r32289) neon/0.28.2" 
server.ip - - [02/May/2010:19:57:31 +0200] "PROPFIND /svn/dotkom/!svn/vcc/default HTTP/1.1" 207 247 "-" "SVN/1.5.1 (r32289) neon/0.28.2" 
server.ip - - [02/May/2010:19:57:31 +0200] "PROPFIND /svn/dotkom/!svn/bln/1650 HTTP/1.1" 207 261 "-" "SVN/1.5.1 (r32289) neon/0.28.2" 
server.ip - - [02/May/2010:19:57:31 +0200] "PROPFIND /svn/dotkom HTTP/1.1" 207 340 "-" "SVN/1.5.1 (r32289) neon/0.28.2" 
server.ip - - [02/May/2010:19:57:31 +0200] "PROPFIND /svn/dotkom/!svn/vcc/default HTTP/1.1" 207 247 "-" "SVN/1.5.1 (r32289) neon/0.28.2" 
server.ip - - [02/May/2010:19:57:31 +0200] "PROPFIND /svn/dotkom/!svn/bln/1650 HTTP/1.1" 207 261 "-" "SVN/1.5.1 (r32289) neon/0.28.2" 
server.ip - - [02/May/2010:19:57:31 +0200] "PROPFIND /svn/dotkom HTTP/1.1" 207 340 "-" "SVN/1.5.1 (r32289) neon/0.28.2" 
server.ip - - [02/May/2010:19:57:31 +0200] "PROPFIND /svn/dotkom/!svn/vcc/default HTTP/1.1" 207 261 "-" "SVN/1.5.1 (r32289) neon/0.28.2" 
server.ip - - [02/May/2010:19:57:31 +0200] "PROPFIND /svn/dotkom/!svn/bc/1650 HTTP/1.1" 207 573 "-" "SVN/1.5.1 (r32289) neon/0.28.2" 
server.ip - - [02/May/2010:19:57:32 +0200] "REPORT /svn/dotkom HTTP/1.1" 200 115 "-" "SVN/1.5.1 (r32289) neon/0.28.2" 
server.ip - - [02/May/2010:19:57:32 +0200] "PROPFIND /svn/dotkom HTTP/1.1" 207 340 "-" "SVN/1.5.1 (r32289) neon/0.28.2" 
server.ip - - [02/May/2010:19:57:32 +0200] "PROPFIND /svn/dotkom/!svn/vcc/default HTTP/1.1" 207 261 "-" "SVN/1.5.1 (r32289) neon/0.28.2" 
server.ip - - [02/May/2010:19:57:32 +0200] "PROPFIND /svn/dotkom/!svn/bc/1650 HTTP/1.1" 207 242 "-" "SVN/1.5.1 (r32289) neon/0.28.2" 
server.ip - - [02/May/2010:19:57:32 +0200] "PROPFIND /svn/dotkom/!svn/bc/1650 HTTP/1.1" 207 760 "-" "SVN/1.5.1 (r32289) neon/0.28.2" 
server.ip - - [02/May/2010:19:57:34 +0200] "OPTIONS /svn/dotkom HTTP/1.1" 200 - "-" "SVN/1.5.1 (r32289) neon/0.28.2" 
server.ip - - [02/May/2010:19:57:34 +0200] "PROPFIND /svn/dotkom HTTP/1.1" 207 340 "-" "SVN/1.5.1 (r32289) neon/0.28.2" 
server.ip - - [02/May/2010:19:57:34 +0200] "PROPFIND /svn/dotkom/!svn/vcc/default HTTP/1.1" 207 247 "-" "SVN/1.5.1 (r32289) neon/0.28.2" 
server.ip - - [02/May/2010:19:57:34 +0200] "PROPFIND /svn/dotkom/!svn/bln/1650 HTTP/1.1" 207 261 "-" "SVN/1.5.1 (r32289) neon/0.28.2" 
server.ip - - [02/May/2010:19:57:34 +0200] "PROPFIND /svn/dotkom HTTP/1.1" 207 340 "-" "SVN/1.5.1 (r32289) neon/0.28.2" 
server.ip - - [02/May/2010:19:57:34 +0200] "PROPFIND /svn/dotkom/!svn/vcc/default HTTP/1.1" 207 247 "-" "SVN/1.5.1 (r32289) neon/0.28.2" 
server.ip - - [02/May/2010:19:57:34 +0200] "PROPFIND /svn/dotkom/!svn/bln/1650 HTTP/1.1" 207 261 "-" "SVN/1.5.1 (r32289) neon/0.28.2" 
server.ip - - [02/May/2010:19:57:34 +0200] "PROPFIND /svn/dotkom HTTP/1.1" 207 340 "-" "SVN/1.5.1 (r32289) neon/0.28.2" 
server.ip - - [02/May/2010:19:57:34 +0200] "PROPFIND /svn/dotkom/!svn/vcc/default HTTP/1.1" 207 247 "-" "SVN/1.5.1 (r32289) neon/0.28.2" 
server.ip - - [02/May/2010:19:57:34 +0200] "PROPFIND /svn/dotkom/!svn/bln/1650 HTTP/1.1" 207 261 "-" "SVN/1.5.1 (r32289) neon/0.28.2" 
server.ip - - [02/May/2010:19:57:34 +0200] "PROPFIND /svn/dotkom HTTP/1.1" 207 340 "-" "SVN/1.5.1 (r32289) neon/0.28.2" 
server.ip - - [02/May/2010:19:57:34 +0200] "PROPFIND /svn/dotkom/!svn/vcc/default HTTP/1.1" 207 261 "-" "SVN/1.5.1 (r32289) neon/0.28.2" 
server.ip - - [02/May/2010:19:57:34 +0200] "REPORT /svn/dotkom/!svn/bc/1650 HTTP/1.1" 200 725 "-" "SVN/1.5.1 (r32289) neon/0.28.2" 
server.ip - - [02/May/2010:19:57:36 +0200] "OPTIONS /svn/dotkom HTTP/1.1" 200 - "-" "SVN/1.5.1 (r32289) neon/0.28.2" 
server.ip - - [02/May/2010:19:57:36 +0200] "PROPFIND /svn/dotkom HTTP/1.1" 207 340 "-" "SVN/1.5.1 (r32289) neon/0.28.2" 
server.ip - - [02/May/2010:19:57:36 +0200] "PROPFIND /svn/dotkom/!svn/vcc/default HTTP/1.1" 207 247 "-" "SVN/1.5.1 (r32289) neon/0.28.2" 
server.ip - - [02/May/2010:19:57:36 +0200] "PROPFIND /svn/dotkom/!svn/bln/1650 HTTP/1.1" 207 261 "-" "SVN/1.5.1 (r32289) neon/0.28.2" 
server.ip - - [02/May/2010:19:57:36 +0200] "PROPFIND /svn/dotkom HTTP/1.1" 207 340 "-" "SVN/1.5.1 (r32289) neon/0.28.2" 
server.ip - - [02/May/2010:19:57:36 +0200] "PROPFIND /svn/dotkom/!svn/vcc/default HTTP/1.1" 207 247 "-" "SVN/1.5.1 (r32289) neon/0.28.2" 
server.ip - - [02/May/2010:19:57:36 +0200] "PROPFIND /svn/dotkom/!svn/bln/1650 HTTP/1.1" 207 261 "-" "SVN/1.5.1 (r32289) neon/0.28.2" 
server.ip - - [02/May/2010:19:57:36 +0200] "PROPFIND /svn/dotkom HTTP/1.1" 207 340 "-" "SVN/1.5.1 (r32289) neon/0.28.2" 
server.ip - - [02/May/2010:19:57:36 +0200] "PROPFIND /svn/dotkom/!svn/vcc/default HTTP/1.1" 207 261 "-" "SVN/1.5.1 (r32289) neon/0.28.2" 
server.ip - - [02/May/2010:19:57:36 +0200] "PROPFIND /svn/dotkom/!svn/bc/1650 HTTP/1.1" 207 350 "-" "SVN/1.5.1 (r32289) neon/0.28.2" 
server.ip - - [02/May/2010:19:57:36 +0200] "PROPFIND /svn/dotkom HTTP/1.1" 207 340 "-" "SVN/1.5.1 (r32289) neon/0.28.2" 
server.ip - - [02/May/2010:19:57:36 +0200] "PROPFIND /svn/dotkom/!svn/vcc/default HTTP/1.1" 207 261 "-" "SVN/1.5.1 (r32289) neon/0.28.2" 
server.ip - - [02/May/2010:19:57:36 +0200] "PROPFIND /svn/dotkom/!svn/bc/1650 HTTP/1.1" 207 242 "-" "SVN/1.5.1 (r32289) neon/0.28.2" 
server.ip - - [02/May/2010:19:57:36 +0200] "PROPFIND /svn/dotkom/!svn/bc/1650 HTTP/1.1" 207 573 "-" "SVN/1.5.1 (r32289) neon/0.28.2" 
my.internet.ip - - [02/May/2010:19:57:30 +0200] "GET /projects/dotkom/repository HTTP/1.1" 304 - "https://company.url/projects/dotkom/repository" "Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.2.5pre) Gecko/20100417 Ubuntu/9.10 (karmic) Firefox/3.5.8pre" 
my.internet.ip - - [02/May/2010:19:57:37 +0200] "GET /stylesheets/application.css?1272717833 HTTP/1.1" 304 - "https://company.url/projects/dotkom/repository" "Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.2.5pre) Gecko/20100417 Ubuntu/9.10 (karmic) Firefox/3.5.8pre" 
my.internet.ip - - [02/May/2010:19:57:37 +0200] "GET /stylesheets/jstoolbar.css?1272717833 HTTP/1.1" 304 - "https://company.url/projects/dotkom/repository" "Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.2.5pre) Gecko/20100417 Ubuntu/9.10 (karmic) Firefox/3.5.8pre" 
my.internet.ip - - [02/May/2010:19:57:37 +0200] "GET /javascripts/repository_navigation.js?1272717833 HTTP/1.1" 304 - "https://company.url/projects/dotkom/repository" "Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.2.5pre) Gecko/20100417 Ubuntu/9.10 (karmic) Firefox/3.5.8pre" 
my.internet.ip - - [02/May/2010:19:57:37 +0200] "GET /javascripts/prototype.js?1272717833 HTTP/1.1" 304 - "https://company.url/projects/dotkom/repository" "Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.2.5pre) Gecko/20100417 Ubuntu/9.10 (karmic) Firefox/3.5.8pre" 
my.internet.ip - - [02/May/2010:19:57:37 +0200] "GET /javascripts/effects.js?1272717833 HTTP/1.1" 304 - "https://company.url/projects/dotkom/repository" "Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.2.5pre) Gecko/20100417 Ubuntu/9.10 (karmic) Firefox/3.5.8pre" 
my.internet.ip - - [02/May/2010:19:57:37 +0200] "GET /javascripts/dragdrop.js?1272717833 HTTP/1.1" 304 - "https://company.url/projects/dotkom/repository" "Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.2.5pre) Gecko/20100417 Ubuntu/9.10 (karmic) Firefox/3.5.8pre" 
my.internet.ip - - [02/May/2010:19:57:37 +0200] "GET /javascripts/controls.js?1272717833 HTTP/1.1" 304 - "https://company.url/projects/dotkom/repository" "Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.2.5pre) Gecko/20100417 Ubuntu/9.10 (karmic) Firefox/3.5.8pre" 
my.internet.ip - - [02/May/2010:19:57:37 +0200] "GET /javascripts/application.js?1272717833 HTTP/1.1" 304 - "https://company.url/projects/dotkom/repository" "Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.2.5pre) Gecko/20100417 Ubuntu/9.10 (karmic) Firefox/3.5.8pre" 
my.internet.ip - - [02/May/2010:19:57:37 +0200] "GET /stylesheets/scm.css?1272717833 HTTP/1.1" 304 - "https://company.url/projects/dotkom/repository" "Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.2.5pre) Gecko/20100417 Ubuntu/9.10 (karmic) Firefox/3.5.8pre"

From redmine log:

Processing RepositoriesController#show (for internet.ip at 2010-05-02 19:57:30) [GET]
  Parameters: {"action"=>"show", "id"=>"dotkom", "controller"=>"repositories"}
Rendering template within layouts/base
Rendering repositories/show
Completed in 6816ms (View: 246, DB: 34) | 200 OK [https://xxx/projects/dotkom/repository]

We can confirm that changesets are loaded into database and setting is correct by:

select * from settings where name~'autofetch';
 id |         name         | value |         updated_on         
----+----------------------+-------+----------------------------
 25 | autofetch_changesets | 0     | 2010-05-02 17:45:26.018804

select count(*) from changes;
 count 
-------
 33073

select count(*) from changesets;
 count 
-------
  3272

RAILS_ENV=production ruby /usr/local/ruby/redmine-0.9.4/script/about

/usr/local/ruby/redmine-0.9.4/script/../config/../vendor/rails/railties/lib/rails/gem_dependency.rb:119:Warning: Gem::Dependency#version_requirements is deprecated and will be removed on or after August 2010.  Use #requirement
About your application's environment
Ruby version              1.8.7 (i686-linux)
RubyGems version          1.3.6
Rack version              1.0
Rails version             2.3.5
Active Record version     2.3.5
Active Resource version   2.3.5
Action Mailer version     2.3.5
Active Support version    2.3.5
Edge Rails revision       unknown
Application root          /usr/local/ruby/redmine-0.9.4
Environment               production
Database adapter          postgresql
Database schema version   20100221100219

Our vhost configuration looks as:

<VirtualHost *:443>
        ServerAdmin dotkom@company.url
        DocumentRoot /usr/local/ruby/redmine-0.9.4/public
        ServerName www.trac.company.url
        ServerAlias trac.company.url

        RequestHeader set X_FORWARDED_PROTO 'https'
        LoadModule passenger_module /usr/local/ruby/lib/ruby/gems/1.8/gems/passenger-2.2.10/ext/apache2/mod_passenger.so
        PassengerRoot /usr/local/ruby/lib/ruby/gems/1.8/gems/passenger-2.2.10
        PassengerRuby /usr/local/ruby/bin/ruby

        <Directory /usr/local/ruby/redmine-0.9.4/public>
                AllowOverride all
                Options -MultiViews
        </Directory>

        PerlLoadModule Apache::Redmine
        PerlLoadModule Authen::Simple::LDAP
        PerlLoadModule IO::Socket:SSL

        Alias /svn /var/svn
        <Location /svn>
            PassengerEnabled Off
                DAV svn
                SVNParentPath "/var/svn" 
                Order deny,allow
                Deny from all
                Satisfy any

                PerlAccessHandler Apache::Authn::Redmine::access_handler
                PerlAuthenHandler Apache::Authn::Redmine::authen_handler
                AuthType Basic
                AuthName "Redmine SVN Repository" 

                #read-only access    
                <Limit GET PROPFIND OPTIONS REPORT>
                        Require valid-user
                        Allow from 127.0.0.1
                        Allow from server.ip
                        # Allow from another-ip
                        Satisfy any
                </Limit>
                # write access
                <LimitExcept GET PROPFIND OPTIONS REPORT>
                        Require valid-user
                </LimitExcept>

                RedmineDSN "DBI:Pg:dbname=redmine;host=db.ip" 
                RedmineDbUser "redmine" 
            RedmineCacheCredsMax "50" 
        </Location>

        <Location "/">
            PassengerEnabled On
        </Location>

        ErrorLog /var/log/apache2/trac-error.log
        LogLevel warn
        CustomLog /var/log/apache2/trac-access.log combined
        ServerSignature On
        SSLCipherSuite HIGH:MEDIUM
</VirtualHost>

Replies (8)

RE: Redmine slow Repository view, everything else fast :/ - Added by Felix Schäfer over 14 years ago

Redmine does only cache the changesets, not the repository structure or content. When you navigate the repository, redmine pulls some svn ls of what you want to view. Depending on how big your repository is (in terms of number of files), this might take a while. Are the repositories on the same server? Could you then try accessing them through file:// instead of https://? Could you time an svn ls from the console on your repository?

RE: Redmine slow Repository view, everything else fast :/ - Added by Roy Sindre Norangshol over 14 years ago

time svn ls https://gfhgfh/svn/dotkom
10 directories

real 0m1.500s
user 0m0.408s
sys 0m0.028s

Rather slow

time svn ls file:///var/svn/dotkom
dirlist

real 0m0.018s
user 0m0.012s
sys 0m0.004s

So, why is DAV Svn so slow?

RE: Redmine slow Repository view, everything else fast :/ - Added by Roy Sindre Norangshol over 14 years ago

Is it possible to configure it so the browser maybe use file:// , but clients can still check out over https+dav?
Still annoying that it is so damn slow tho. Is it normal?

RE: Redmine slow Repository view, everything else fast :/ - Added by Felix Schäfer over 14 years ago

Well, the https:// checkout is not bound to the repository being configured with https://, so yes, you can configure your repositories to be accessed over file:// from redmine and https from your users. As to why the webdav thing is slow: I don't know, I have 2 setups with svn repositories accessed by redmine over file:// and served to clients over https://, everything authenticated against LDAP, and everything works great. Any more info on your setup?

RE: Redmine slow Repository view, everything else fast :/ - Added by Roy Sindre Norangshol over 14 years ago

Users are authenticated against ldap, and I've made sure to setup cache so it doesn't hammer the ldap server with bind-auth's.

And redmine is able to access the repository over file:// as they are on the same server. I'm wondering why it isn't doing it already, since:

select * from repositories shows URL to https://dav one, and root_url is set to file:// . Shouldn't that be right?

I'm also available over irc at freenode as Rockj

RE: Redmine slow Repository view, everything else fast :/ - Added by Roy Sindre Norangshol over 14 years ago

And from the one who pointed out online at IRC that I'd have to make sure my repository tab in project config should say file:// :

What about repository permissions then? I'm fine with redmine accessing it by file://, but not users.

Sure those who have shell users (which is some of us) can have permissions set by owner in linux, but there is still an issue for those who can only access the repository from https://dav since Redmine is working out who has access to which repository.

RE: Redmine slow Repository view, everything else fast :/ - Added by Roy Sindre Norangshol over 14 years ago

Correction, settings repository tab to actually use file:// of course works like a charm.

Since it is the apache2 and it's Redmine.pm module which takes cares of authentication.

Anyway, that at least fixed the slow Repository view!

Accessing https://DAV is still mad slow, so if anyone have tips on what to do here, I don't mind getting some tips on what to do to speed it up. (should I create a new thread for that problem?)

RE: Redmine slow Repository view, everything else fast :/ - Added by Felix Schäfer over 14 years ago

Create a new thread or even file a bug report wrt the slowness of the DAV access, I'd recommend testing access without the redmine.pm authentication first (set up a basic auth to test) to really make sure the redmine.pm is to blame.

(1-8/8)

Project

General

Profile

Redmine