Project

General

Profile

Forums » Help »

New project using existing 1.5Gb file:/// svn repository crashes Redmine.

Added by Chris Dew about 16 years ago

I've created a new project, within Redmine r1969. It uses a pre-existing 1.5Gb repository containing 4000+ commits.

When I click on the 'Repository' link from within the project, the browser freezes for 5-10 minutes; it then produces the error below.

It seems that Redmine is looking at the logs, and trying to write that information into it's mysql database. It does this in chunks of 200 commits at a time. The commits from r400-r600 seem to take too long and cause the mysql timeout. During the import, it is the 'ruby' process which is using 90-100% cpu, not the 'svn log' command.

Is this a bug, or is there some parameter which I should tune (in either Redmine or MySQL)?

Thanks,

Chris.

ActiveRecord::StatementInvalid in RepositoriesController#show

Mysql::Error: #HY000Lock wait timeout exceeded; try restarting transaction: INSERT INTO `changesets` (`commit_date`, `committed_on`, `comments`, `revision`, `scmid`, `repository_id`, `committer`) VALUES('2007-04-26 15:54:03', '2007-04-26 15:54:03', 'Made the boot script delete dansguardian.pid.', '201', NULL, 2, 'dan')

RAILS_ROOT: /home/chris/redmine
Application Trace | Framework Trace | Full Trace

/var/lib/gems/1.8/gems/activerecord-2.1.2/lib/active_record/connection_adapters/abstract_adapter.rb:147:in `log'
/var/lib/gems/1.8/gems/activerecord-2.1.2/lib/active_record/connection_adapters/mysql_adapter.rb:302:in `execute'
/var/lib/gems/1.8/gems/activerecord-2.1.2/lib/active_record/connection_adapters/abstract/database_statements.rb:161:in `insert_sql'
/var/lib/gems/1.8/gems/activerecord-2.1.2/lib/active_record/connection_adapters/mysql_adapter.rb:312:in `insert_sql'
/var/lib/gems/1.8/gems/activerecord-2.1.2/lib/active_record/connection_adapters/abstract/database_statements.rb:44:in `insert_without_query_dirty'
/var/lib/gems/1.8/gems/activerecord-2.1.2/lib/active_record/connection_adapters/abstract/query_cache.rb:19:in `insert'
/var/lib/gems/1.8/gems/activerecord-2.1.2/lib/active_record/base.rb:2517:in `create_without_callbacks'
/var/lib/gems/1.8/gems/activerecord-2.1.2/lib/active_record/callbacks.rb:220:in `create_without_timestamps'
/var/lib/gems/1.8/gems/activerecord-2.1.2/lib/active_record/timestamp.rb:29:in `create'
/var/lib/gems/1.8/gems/activerecord-2.1.2/lib/active_record/base.rb:2483:in `create_or_update_without_callbacks'
/var/lib/gems/1.8/gems/activerecord-2.1.2/lib/active_record/callbacks.rb:207:in `create_or_update'
/var/lib/gems/1.8/gems/activerecord-2.1.2/lib/active_record/base.rb:2211:in `save_without_validation'
/var/lib/gems/1.8/gems/activerecord-2.1.2/lib/active_record/validations.rb:911:in `save_without_dirty'
/var/lib/gems/1.8/gems/activerecord-2.1.2/lib/active_record/dirty.rb:75:in `save_without_transactions'
/var/lib/gems/1.8/gems/activerecord-2.1.2/lib/active_record/transactions.rb:106:in `save'
/var/lib/gems/1.8/gems/activerecord-2.1.2/lib/active_record/connection_adapters/abstract/database_statements.rb:66:in `transaction'
/var/lib/gems/1.8/gems/activerecord-2.1.2/lib/active_record/transactions.rb:79:in `transaction'
/var/lib/gems/1.8/gems/activerecord-2.1.2/lib/active_record/transactions.rb:98:in `transaction'
/var/lib/gems/1.8/gems/activerecord-2.1.2/lib/active_record/transactions.rb:106:in `save'
/var/lib/gems/1.8/gems/activerecord-2.1.2/lib/active_record/transactions.rb:118:in `rollback_active_record_state!'
/var/lib/gems/1.8/gems/activerecord-2.1.2/lib/active_record/transactions.rb:106:in `save'
/var/lib/gems/1.8/gems/activerecord-2.1.2/lib/active_record/base.rb:644:in `create'
app/models/repository/subversion.rb:59:in `fetch_changesets'
app/models/repository/subversion.rb:58:in `reverse_each'
app/models/repository/subversion.rb:58:in `fetch_changesets'
/var/lib/gems/1.8/gems/activerecord-2.1.2/lib/active_record/connection_adapters/abstract/database_statements.rb:66:in `transaction'
/var/lib/gems/1.8/gems/activerecord-2.1.2/lib/active_record/transactions.rb:79:in `transaction'
/var/lib/gems/1.8/gems/activerecord-2.1.2/lib/active_record/transactions.rb:98:in `transaction'
app/models/repository/subversion.rb:57:in `fetch_changesets'

/var/lib/gems/1.8/gems/activerecord-2.1.2/lib/active_record/associations/association_proxy.rb:173:in `send'
/var/lib/gems/1.8/gems/activerecord-2.1.2/lib/active_record/associations/association_proxy.rb:173:in `method_missing'
app/controllers/repositories_controller.rb:54:in `show'
/var/lib/gems/1.8/gems/actionpack-2.1.2/lib/action_controller/base.rb:1166:in `send'
/var/lib/gems/1.8/gems/actionpack-2.1.2/lib/action_controller/base.rb:1166:in `perform_action_without_filters'
/var/lib/gems/1.8/gems/actionpack-2.1.2/lib/action_controller/filters.rb:579:in `call_filters'
/var/lib/gems/1.8/gems/actionpack-2.1.2/lib/action_controller/filters.rb:572:in `perform_action_without_benchmark'
/var/lib/gems/1.8/gems/actionpack-2.1.2/lib/action_controller/benchmarking.rb:68:in `perform_action_without_rescue'
/var/lib/gems/1.8/gems/actionpack-2.1.2/lib/action_controller/benchmarking.rb:68:in `perform_action_without_rescue'
/var/lib/gems/1.8/gems/actionpack-2.1.2/lib/action_controller/rescue.rb:201:in `perform_action_without_caching'
/var/lib/gems/1.8/gems/actionpack-2.1.2/lib/action_controller/caching/sql_cache.rb:13:in `perform_action'
/var/lib/gems/1.8/gems/activerecord-2.1.2/lib/active_record/connection_adapters/abstract/query_cache.rb:33:in `cache'
/var/lib/gems/1.8/gems/activerecord-2.1.2/lib/active_record/query_cache.rb:8:in `cache'
/var/lib/gems/1.8/gems/actionpack-2.1.2/lib/action_controller/caching/sql_cache.rb:12:in `perform_action'
/var/lib/gems/1.8/gems/actionpack-2.1.2/lib/action_controller/base.rb:529:in `send'
/var/lib/gems/1.8/gems/actionpack-2.1.2/lib/action_controller/base.rb:529:in `process_without_filters'
/var/lib/gems/1.8/gems/actionpack-2.1.2/lib/action_controller/filters.rb:568:in `process_without_session_management_support'
/var/lib/gems/1.8/gems/actionpack-2.1.2/lib/action_controller/session_management.rb:130:in `process'
/var/lib/gems/1.8/gems/actionpack-2.1.2/lib/action_controller/base.rb:389:in `process'
/var/lib/gems/1.8/gems/rails-2.1.2/lib/webrick_server.rb:78:in `service'
/var/lib/gems/1.8/gems/rails-2.1.2/lib/commands/servers/webrick.rb:66
/var/lib/gems/1.8/gems/activesupport-2.1.2/lib/active_support/dependencies.rb:510:in `require'
/var/lib/gems/1.8/gems/activesupport-2.1.2/lib/active_support/dependencies.rb:355:in `new_constants_in'
/var/lib/gems/1.8/gems/activesupport-2.1.2/lib/active_support/dependencies.rb:510:in `require'
/var/lib/gems/1.8/gems/rails-2.1.2/lib/commands/server.rb:39

/usr/lib/ruby/1.8/benchmark.rb:293:in `measure'
/var/lib/gems/1.8/gems/actionpack-2.1.2/lib/action_controller/dispatcher.rb:149:in `handle_request'
/var/lib/gems/1.8/gems/actionpack-2.1.2/lib/action_controller/dispatcher.rb:107:in `dispatch'
/var/lib/gems/1.8/gems/actionpack-2.1.2/lib/action_controller/dispatcher.rb:104:in `synchronize'
/var/lib/gems/1.8/gems/actionpack-2.1.2/lib/action_controller/dispatcher.rb:104:in `dispatch'
/var/lib/gems/1.8/gems/actionpack-2.1.2/lib/action_controller/dispatcher.rb:120:in `dispatch_cgi'
/var/lib/gems/1.8/gems/actionpack-2.1.2/lib/action_controller/dispatcher.rb:35:in `dispatch'
/var/lib/gems/1.8/gems/rails-2.1.2/lib/webrick_server.rb:112:in `handle_dispatch'
/usr/lib/ruby/1.8/webrick/httpserver.rb:104:in `service'
/usr/lib/ruby/1.8/webrick/httpserver.rb:65:in `run'
/usr/lib/ruby/1.8/webrick/server.rb:173:in `start_thread'
/usr/lib/ruby/1.8/webrick/server.rb:162:in `start'
/usr/lib/ruby/1.8/webrick/server.rb:162:in `start_thread'
/usr/lib/ruby/1.8/webrick/server.rb:95:in `start'
/usr/lib/ruby/1.8/webrick/server.rb:92:in `each'
/usr/lib/ruby/1.8/webrick/server.rb:92:in `start'
/usr/lib/ruby/1.8/webrick/server.rb:23:in `start'
/usr/lib/ruby/1.8/webrick/server.rb:82:in `start'
/var/lib/gems/1.8/gems/rails-2.1.2/lib/webrick_server.rb:62:in `dispatch'
/usr/lib/ruby/1.8/rubygems/custom_require.rb:27:in `gem_original_require'
/usr/lib/ruby/1.8/rubygems/custom_require.rb:27:in `require'
/usr/lib/ruby/1.8/rubygems/custom_require.rb:27:in `gem_original_require'
/usr/lib/ruby/1.8/rubygems/custom_require.rb:27:in `require'
script/server:3

/var/lib/gems/1.8/gems/activerecord-2.1.2/lib/active_record/connection_adapters/abstract_adapter.rb:147:in `log'
/var/lib/gems/1.8/gems/activerecord-2.1.2/lib/active_record/connection_adapters/mysql_adapter.rb:302:in `execute'
/var/lib/gems/1.8/gems/activerecord-2.1.2/lib/active_record/connection_adapters/abstract/database_statements.rb:161:in `insert_sql'
/var/lib/gems/1.8/gems/activerecord-2.1.2/lib/active_record/connection_adapters/mysql_adapter.rb:312:in `insert_sql'
/var/lib/gems/1.8/gems/activerecord-2.1.2/lib/active_record/connection_adapters/abstract/database_statements.rb:44:in `insert_without_query_dirty'
/var/lib/gems/1.8/gems/activerecord-2.1.2/lib/active_record/connection_adapters/abstract/query_cache.rb:19:in `insert'
/var/lib/gems/1.8/gems/activerecord-2.1.2/lib/active_record/base.rb:2517:in `create_without_callbacks'
/var/lib/gems/1.8/gems/activerecord-2.1.2/lib/active_record/callbacks.rb:220:in `create_without_timestamps'
/var/lib/gems/1.8/gems/activerecord-2.1.2/lib/active_record/timestamp.rb:29:in `create'
/var/lib/gems/1.8/gems/activerecord-2.1.2/lib/active_record/base.rb:2483:in `create_or_update_without_callbacks'
/var/lib/gems/1.8/gems/activerecord-2.1.2/lib/active_record/callbacks.rb:207:in `create_or_update'
/var/lib/gems/1.8/gems/activerecord-2.1.2/lib/active_record/base.rb:2211:in `save_without_validation'
/var/lib/gems/1.8/gems/activerecord-2.1.2/lib/active_record/validations.rb:911:in `save_without_dirty'
/var/lib/gems/1.8/gems/activerecord-2.1.2/lib/active_record/dirty.rb:75:in `save_without_transactions'
/var/lib/gems/1.8/gems/activerecord-2.1.2/lib/active_record/transactions.rb:106:in `save'
/var/lib/gems/1.8/gems/activerecord-2.1.2/lib/active_record/connection_adapters/abstract/database_statements.rb:66:in `transaction'
/var/lib/gems/1.8/gems/activerecord-2.1.2/lib/active_record/transactions.rb:79:in `transaction'
/var/lib/gems/1.8/gems/activerecord-2.1.2/lib/active_record/transactions.rb:98:in `transaction'
/var/lib/gems/1.8/gems/activerecord-2.1.2/lib/active_record/transactions.rb:106:in `save'
/var/lib/gems/1.8/gems/activerecord-2.1.2/lib/active_record/transactions.rb:118:in `rollback_active_record_state!'
/var/lib/gems/1.8/gems/activerecord-2.1.2/lib/active_record/transactions.rb:106:in `save'
/var/lib/gems/1.8/gems/activerecord-2.1.2/lib/active_record/base.rb:644:in `create'
app/models/repository/subversion.rb:59:in `fetch_changesets'
app/models/repository/subversion.rb:58:in `reverse_each'
app/models/repository/subversion.rb:58:in `fetch_changesets'
/var/lib/gems/1.8/gems/activerecord-2.1.2/lib/active_record/connection_adapters/abstract/database_statements.rb:66:in `transaction'
/var/lib/gems/1.8/gems/activerecord-2.1.2/lib/active_record/transactions.rb:79:in `transaction'
/var/lib/gems/1.8/gems/activerecord-2.1.2/lib/active_record/transactions.rb:98:in `transaction'
app/models/repository/subversion.rb:57:in `fetch_changesets'
/var/lib/gems/1.8/gems/activerecord-2.1.2/lib/active_record/associations/association_proxy.rb:173:in `send'
/var/lib/gems/1.8/gems/activerecord-2.1.2/lib/active_record/associations/association_proxy.rb:173:in `method_missing'
app/controllers/repositories_controller.rb:54:in `show'
/var/lib/gems/1.8/gems/actionpack-2.1.2/lib/action_controller/base.rb:1166:in `send'
/var/lib/gems/1.8/gems/actionpack-2.1.2/lib/action_controller/base.rb:1166:in `perform_action_without_filters'
/var/lib/gems/1.8/gems/actionpack-2.1.2/lib/action_controller/filters.rb:579:in `call_filters'
/var/lib/gems/1.8/gems/actionpack-2.1.2/lib/action_controller/filters.rb:572:in `perform_action_without_benchmark'
/var/lib/gems/1.8/gems/actionpack-2.1.2/lib/action_controller/benchmarking.rb:68:in `perform_action_without_rescue'
/usr/lib/ruby/1.8/benchmark.rb:293:in `measure'
/var/lib/gems/1.8/gems/actionpack-2.1.2/lib/action_controller/benchmarking.rb:68:in `perform_action_without_rescue'
/var/lib/gems/1.8/gems/actionpack-2.1.2/lib/action_controller/rescue.rb:201:in `perform_action_without_caching'
/var/lib/gems/1.8/gems/actionpack-2.1.2/lib/action_controller/caching/sql_cache.rb:13:in `perform_action'
/var/lib/gems/1.8/gems/activerecord-2.1.2/lib/active_record/connection_adapters/abstract/query_cache.rb:33:in `cache'
/var/lib/gems/1.8/gems/activerecord-2.1.2/lib/active_record/query_cache.rb:8:in `cache'
/var/lib/gems/1.8/gems/actionpack-2.1.2/lib/action_controller/caching/sql_cache.rb:12:in `perform_action'
/var/lib/gems/1.8/gems/actionpack-2.1.2/lib/action_controller/base.rb:529:in `send'
/var/lib/gems/1.8/gems/actionpack-2.1.2/lib/action_controller/base.rb:529:in `process_without_filters'
/var/lib/gems/1.8/gems/actionpack-2.1.2/lib/action_controller/filters.rb:568:in `process_without_session_management_support'
/var/lib/gems/1.8/gems/actionpack-2.1.2/lib/action_controller/session_management.rb:130:in `process'
/var/lib/gems/1.8/gems/actionpack-2.1.2/lib/action_controller/base.rb:389:in `process'
/var/lib/gems/1.8/gems/actionpack-2.1.2/lib/action_controller/dispatcher.rb:149:in `handle_request'
/var/lib/gems/1.8/gems/actionpack-2.1.2/lib/action_controller/dispatcher.rb:107:in `dispatch'
/var/lib/gems/1.8/gems/actionpack-2.1.2/lib/action_controller/dispatcher.rb:104:in `synchronize'
/var/lib/gems/1.8/gems/actionpack-2.1.2/lib/action_controller/dispatcher.rb:104:in `dispatch'
/var/lib/gems/1.8/gems/actionpack-2.1.2/lib/action_controller/dispatcher.rb:120:in `dispatch_cgi'
/var/lib/gems/1.8/gems/actionpack-2.1.2/lib/action_controller/dispatcher.rb:35:in `dispatch'
/var/lib/gems/1.8/gems/rails-2.1.2/lib/webrick_server.rb:112:in `handle_dispatch'
/var/lib/gems/1.8/gems/rails-2.1.2/lib/webrick_server.rb:78:in `service'
/usr/lib/ruby/1.8/webrick/httpserver.rb:104:in `service'
/usr/lib/ruby/1.8/webrick/httpserver.rb:65:in `run'
/usr/lib/ruby/1.8/webrick/server.rb:173:in `start_thread'
/usr/lib/ruby/1.8/webrick/server.rb:162:in `start'
/usr/lib/ruby/1.8/webrick/server.rb:162:in `start_thread'
/usr/lib/ruby/1.8/webrick/server.rb:95:in `start'
/usr/lib/ruby/1.8/webrick/server.rb:92:in `each'
/usr/lib/ruby/1.8/webrick/server.rb:92:in `start'
/usr/lib/ruby/1.8/webrick/server.rb:23:in `start'
/usr/lib/ruby/1.8/webrick/server.rb:82:in `start'
/var/lib/gems/1.8/gems/rails-2.1.2/lib/webrick_server.rb:62:in `dispatch'
/var/lib/gems/1.8/gems/rails-2.1.2/lib/commands/servers/webrick.rb:66
/usr/lib/ruby/1.8/rubygems/custom_require.rb:27:in `gem_original_require'
/usr/lib/ruby/1.8/rubygems/custom_require.rb:27:in `require'
/var/lib/gems/1.8/gems/activesupport-2.1.2/lib/active_support/dependencies.rb:510:in `require'
/var/lib/gems/1.8/gems/activesupport-2.1.2/lib/active_support/dependencies.rb:355:in `new_constants_in'
/var/lib/gems/1.8/gems/activesupport-2.1.2/lib/active_support/dependencies.rb:510:in `require'
/var/lib/gems/1.8/gems/rails-2.1.2/lib/commands/server.rb:39
/usr/lib/ruby/1.8/rubygems/custom_require.rb:27:in `gem_original_require'
/usr/lib/ruby/1.8/rubygems/custom_require.rb:27:in `require'
script/server:3

Request

Parameters:

{"id"=>"app"}

Show session dump

--- 
userslist_sort: 
  :key: login
  :order: asc
boardsindex_sort: 
  :key: messages.updated_on
  :order: desc
projectslist_files_sort: 
  :key: attachments.filename
  :order: asc
:user_id: 
timelogdetails_sort: 
  :key: spent_on
  :order: desc
:query: 
  :project_id: 1
  :filters: 
    tracker_id: 
      :values: 
      - "1" 
      :operator: "=" 
    status_id: 
      :values: 
      - "" 
      :operator: o
adminprojects_sort: 
  :key: name
  :order: asc
issuesindex_sort: 
  :key: issues.id
  :order: desc
flash: !map:ActionController::Flash::FlashHash {}

Response

Headers:

{"cookie"=>[],
 "Cache-Control"=>"no-cache"}

(Edit (T. Lecavelier): dump format )


Replies (4)

RE: New project using existing 1.5Gb file:/// svn repository crashes Redmine. - Added by Karl Heinz Marbaise about 16 years ago

Hi,

i would suggest to do the repository scanning for the first time by hand:
Take a look at the FAQ

Kind regards
Karl Heinz Marbase

RE: New project using existing 1.5Gb file:/// svn repository crashes Redmine. - Added by Chris Dew about 16 years ago

Thanks for your reply - I felt a little ashamed at being pointed to the FAQ, but at least the question there was not my question ;-)

Here's the link.

http://www.redmine.org/wiki/redmine/FAQ#Commits-dont-show-up-in-the-activity-until-I-click-on-Repository

All the best,

Chris.

RE: New project using existing 1.5Gb file:/// svn repository crashes Redmine. - Added by Karl Heinz Marbaise about 16 years ago

Hi Chris,
i wouldn't make you feeling not good sorry...

Thanks for your reply - I felt a little ashamed at being pointed to the FAQ, but at least the question there was not my question ;-)

I think based on this size of your repository it takes simply too long so you should scan your repository by hand before...

http://www.redmine.org/wiki/redmine/FAQ#Commits-dont-show-up-in-the-activity-until-I-click-on-Repository

Sorry for the wrong line....

Kind regards
Karl Heinz Marbaise

RE: New project using existing 1.5Gb file:/// svn repository crashes Redmine. - Added by Fernando Kosh over 15 years ago

Hi Chris and all colleagues,

I have the same problem running Redmine 0.8.2 with a repository containing 2000+ commits.

I follow these instructions: 

http://www.redmine.org/wiki/redmine/FAQ#Commits-dont-show-up-in-the-activity-until-I-click-on-Repository

And got a the error again. But, looking into MySQL, I see parts of data from repository. So, I repeat the proccess again and again, always having errors, and every time I see more data in data base. After repeat this step many times, it finally ends fecthing all data and works better now.

It solv my problem, but I want to know if is there other way to do the same.

    (1-4/4)