New project using existing 1.5Gb file:/// svn repository crashes Redmine.
Added by Chris Dew almost 16 years ago
I've created a new project, within Redmine r1969. It uses a pre-existing 1.5Gb repository containing 4000+ commits.
When I click on the 'Repository' link from within the project, the browser freezes for 5-10 minutes; it then produces the error below.
It seems that Redmine is looking at the logs, and trying to write that information into it's mysql database. It does this in chunks of 200 commits at a time. The commits from r400-r600 seem to take too long and cause the mysql timeout. During the import, it is the 'ruby' process which is using 90-100% cpu, not the 'svn log' command.
Is this a bug, or is there some parameter which I should tune (in either Redmine or MySQL)?
Thanks,
Chris.
ActiveRecord::StatementInvalid in RepositoriesController#show Mysql::Error: #HY000Lock wait timeout exceeded; try restarting transaction: INSERT INTO `changesets` (`commit_date`, `committed_on`, `comments`, `revision`, `scmid`, `repository_id`, `committer`) VALUES('2007-04-26 15:54:03', '2007-04-26 15:54:03', 'Made the boot script delete dansguardian.pid.', '201', NULL, 2, 'dan') RAILS_ROOT: /home/chris/redmine Application Trace | Framework Trace | Full Trace /var/lib/gems/1.8/gems/activerecord-2.1.2/lib/active_record/connection_adapters/abstract_adapter.rb:147:in `log' /var/lib/gems/1.8/gems/activerecord-2.1.2/lib/active_record/connection_adapters/mysql_adapter.rb:302:in `execute' /var/lib/gems/1.8/gems/activerecord-2.1.2/lib/active_record/connection_adapters/abstract/database_statements.rb:161:in `insert_sql' /var/lib/gems/1.8/gems/activerecord-2.1.2/lib/active_record/connection_adapters/mysql_adapter.rb:312:in `insert_sql' /var/lib/gems/1.8/gems/activerecord-2.1.2/lib/active_record/connection_adapters/abstract/database_statements.rb:44:in `insert_without_query_dirty' /var/lib/gems/1.8/gems/activerecord-2.1.2/lib/active_record/connection_adapters/abstract/query_cache.rb:19:in `insert' /var/lib/gems/1.8/gems/activerecord-2.1.2/lib/active_record/base.rb:2517:in `create_without_callbacks' /var/lib/gems/1.8/gems/activerecord-2.1.2/lib/active_record/callbacks.rb:220:in `create_without_timestamps' /var/lib/gems/1.8/gems/activerecord-2.1.2/lib/active_record/timestamp.rb:29:in `create' /var/lib/gems/1.8/gems/activerecord-2.1.2/lib/active_record/base.rb:2483:in `create_or_update_without_callbacks' /var/lib/gems/1.8/gems/activerecord-2.1.2/lib/active_record/callbacks.rb:207:in `create_or_update' /var/lib/gems/1.8/gems/activerecord-2.1.2/lib/active_record/base.rb:2211:in `save_without_validation' /var/lib/gems/1.8/gems/activerecord-2.1.2/lib/active_record/validations.rb:911:in `save_without_dirty' /var/lib/gems/1.8/gems/activerecord-2.1.2/lib/active_record/dirty.rb:75:in `save_without_transactions' /var/lib/gems/1.8/gems/activerecord-2.1.2/lib/active_record/transactions.rb:106:in `save' /var/lib/gems/1.8/gems/activerecord-2.1.2/lib/active_record/connection_adapters/abstract/database_statements.rb:66:in `transaction' /var/lib/gems/1.8/gems/activerecord-2.1.2/lib/active_record/transactions.rb:79:in `transaction' /var/lib/gems/1.8/gems/activerecord-2.1.2/lib/active_record/transactions.rb:98:in `transaction' /var/lib/gems/1.8/gems/activerecord-2.1.2/lib/active_record/transactions.rb:106:in `save' /var/lib/gems/1.8/gems/activerecord-2.1.2/lib/active_record/transactions.rb:118:in `rollback_active_record_state!' /var/lib/gems/1.8/gems/activerecord-2.1.2/lib/active_record/transactions.rb:106:in `save' /var/lib/gems/1.8/gems/activerecord-2.1.2/lib/active_record/base.rb:644:in `create' app/models/repository/subversion.rb:59:in `fetch_changesets' app/models/repository/subversion.rb:58:in `reverse_each' app/models/repository/subversion.rb:58:in `fetch_changesets' /var/lib/gems/1.8/gems/activerecord-2.1.2/lib/active_record/connection_adapters/abstract/database_statements.rb:66:in `transaction' /var/lib/gems/1.8/gems/activerecord-2.1.2/lib/active_record/transactions.rb:79:in `transaction' /var/lib/gems/1.8/gems/activerecord-2.1.2/lib/active_record/transactions.rb:98:in `transaction' app/models/repository/subversion.rb:57:in `fetch_changesets' /var/lib/gems/1.8/gems/activerecord-2.1.2/lib/active_record/associations/association_proxy.rb:173:in `send' /var/lib/gems/1.8/gems/activerecord-2.1.2/lib/active_record/associations/association_proxy.rb:173:in `method_missing' app/controllers/repositories_controller.rb:54:in `show' /var/lib/gems/1.8/gems/actionpack-2.1.2/lib/action_controller/base.rb:1166:in `send' /var/lib/gems/1.8/gems/actionpack-2.1.2/lib/action_controller/base.rb:1166:in `perform_action_without_filters' /var/lib/gems/1.8/gems/actionpack-2.1.2/lib/action_controller/filters.rb:579:in `call_filters' /var/lib/gems/1.8/gems/actionpack-2.1.2/lib/action_controller/filters.rb:572:in `perform_action_without_benchmark' /var/lib/gems/1.8/gems/actionpack-2.1.2/lib/action_controller/benchmarking.rb:68:in `perform_action_without_rescue' /var/lib/gems/1.8/gems/actionpack-2.1.2/lib/action_controller/benchmarking.rb:68:in `perform_action_without_rescue' /var/lib/gems/1.8/gems/actionpack-2.1.2/lib/action_controller/rescue.rb:201:in `perform_action_without_caching' /var/lib/gems/1.8/gems/actionpack-2.1.2/lib/action_controller/caching/sql_cache.rb:13:in `perform_action' /var/lib/gems/1.8/gems/activerecord-2.1.2/lib/active_record/connection_adapters/abstract/query_cache.rb:33:in `cache' /var/lib/gems/1.8/gems/activerecord-2.1.2/lib/active_record/query_cache.rb:8:in `cache' /var/lib/gems/1.8/gems/actionpack-2.1.2/lib/action_controller/caching/sql_cache.rb:12:in `perform_action' /var/lib/gems/1.8/gems/actionpack-2.1.2/lib/action_controller/base.rb:529:in `send' /var/lib/gems/1.8/gems/actionpack-2.1.2/lib/action_controller/base.rb:529:in `process_without_filters' /var/lib/gems/1.8/gems/actionpack-2.1.2/lib/action_controller/filters.rb:568:in `process_without_session_management_support' /var/lib/gems/1.8/gems/actionpack-2.1.2/lib/action_controller/session_management.rb:130:in `process' /var/lib/gems/1.8/gems/actionpack-2.1.2/lib/action_controller/base.rb:389:in `process' /var/lib/gems/1.8/gems/rails-2.1.2/lib/webrick_server.rb:78:in `service' /var/lib/gems/1.8/gems/rails-2.1.2/lib/commands/servers/webrick.rb:66 /var/lib/gems/1.8/gems/activesupport-2.1.2/lib/active_support/dependencies.rb:510:in `require' /var/lib/gems/1.8/gems/activesupport-2.1.2/lib/active_support/dependencies.rb:355:in `new_constants_in' /var/lib/gems/1.8/gems/activesupport-2.1.2/lib/active_support/dependencies.rb:510:in `require' /var/lib/gems/1.8/gems/rails-2.1.2/lib/commands/server.rb:39 /usr/lib/ruby/1.8/benchmark.rb:293:in `measure' /var/lib/gems/1.8/gems/actionpack-2.1.2/lib/action_controller/dispatcher.rb:149:in `handle_request' /var/lib/gems/1.8/gems/actionpack-2.1.2/lib/action_controller/dispatcher.rb:107:in `dispatch' /var/lib/gems/1.8/gems/actionpack-2.1.2/lib/action_controller/dispatcher.rb:104:in `synchronize' /var/lib/gems/1.8/gems/actionpack-2.1.2/lib/action_controller/dispatcher.rb:104:in `dispatch' /var/lib/gems/1.8/gems/actionpack-2.1.2/lib/action_controller/dispatcher.rb:120:in `dispatch_cgi' /var/lib/gems/1.8/gems/actionpack-2.1.2/lib/action_controller/dispatcher.rb:35:in `dispatch' /var/lib/gems/1.8/gems/rails-2.1.2/lib/webrick_server.rb:112:in `handle_dispatch' /usr/lib/ruby/1.8/webrick/httpserver.rb:104:in `service' /usr/lib/ruby/1.8/webrick/httpserver.rb:65:in `run' /usr/lib/ruby/1.8/webrick/server.rb:173:in `start_thread' /usr/lib/ruby/1.8/webrick/server.rb:162:in `start' /usr/lib/ruby/1.8/webrick/server.rb:162:in `start_thread' /usr/lib/ruby/1.8/webrick/server.rb:95:in `start' /usr/lib/ruby/1.8/webrick/server.rb:92:in `each' /usr/lib/ruby/1.8/webrick/server.rb:92:in `start' /usr/lib/ruby/1.8/webrick/server.rb:23:in `start' /usr/lib/ruby/1.8/webrick/server.rb:82:in `start' /var/lib/gems/1.8/gems/rails-2.1.2/lib/webrick_server.rb:62:in `dispatch' /usr/lib/ruby/1.8/rubygems/custom_require.rb:27:in `gem_original_require' /usr/lib/ruby/1.8/rubygems/custom_require.rb:27:in `require' /usr/lib/ruby/1.8/rubygems/custom_require.rb:27:in `gem_original_require' /usr/lib/ruby/1.8/rubygems/custom_require.rb:27:in `require' script/server:3 /var/lib/gems/1.8/gems/activerecord-2.1.2/lib/active_record/connection_adapters/abstract_adapter.rb:147:in `log' /var/lib/gems/1.8/gems/activerecord-2.1.2/lib/active_record/connection_adapters/mysql_adapter.rb:302:in `execute' /var/lib/gems/1.8/gems/activerecord-2.1.2/lib/active_record/connection_adapters/abstract/database_statements.rb:161:in `insert_sql' /var/lib/gems/1.8/gems/activerecord-2.1.2/lib/active_record/connection_adapters/mysql_adapter.rb:312:in `insert_sql' /var/lib/gems/1.8/gems/activerecord-2.1.2/lib/active_record/connection_adapters/abstract/database_statements.rb:44:in `insert_without_query_dirty' /var/lib/gems/1.8/gems/activerecord-2.1.2/lib/active_record/connection_adapters/abstract/query_cache.rb:19:in `insert' /var/lib/gems/1.8/gems/activerecord-2.1.2/lib/active_record/base.rb:2517:in `create_without_callbacks' /var/lib/gems/1.8/gems/activerecord-2.1.2/lib/active_record/callbacks.rb:220:in `create_without_timestamps' /var/lib/gems/1.8/gems/activerecord-2.1.2/lib/active_record/timestamp.rb:29:in `create' /var/lib/gems/1.8/gems/activerecord-2.1.2/lib/active_record/base.rb:2483:in `create_or_update_without_callbacks' /var/lib/gems/1.8/gems/activerecord-2.1.2/lib/active_record/callbacks.rb:207:in `create_or_update' /var/lib/gems/1.8/gems/activerecord-2.1.2/lib/active_record/base.rb:2211:in `save_without_validation' /var/lib/gems/1.8/gems/activerecord-2.1.2/lib/active_record/validations.rb:911:in `save_without_dirty' /var/lib/gems/1.8/gems/activerecord-2.1.2/lib/active_record/dirty.rb:75:in `save_without_transactions' /var/lib/gems/1.8/gems/activerecord-2.1.2/lib/active_record/transactions.rb:106:in `save' /var/lib/gems/1.8/gems/activerecord-2.1.2/lib/active_record/connection_adapters/abstract/database_statements.rb:66:in `transaction' /var/lib/gems/1.8/gems/activerecord-2.1.2/lib/active_record/transactions.rb:79:in `transaction' /var/lib/gems/1.8/gems/activerecord-2.1.2/lib/active_record/transactions.rb:98:in `transaction' /var/lib/gems/1.8/gems/activerecord-2.1.2/lib/active_record/transactions.rb:106:in `save' /var/lib/gems/1.8/gems/activerecord-2.1.2/lib/active_record/transactions.rb:118:in `rollback_active_record_state!' /var/lib/gems/1.8/gems/activerecord-2.1.2/lib/active_record/transactions.rb:106:in `save' /var/lib/gems/1.8/gems/activerecord-2.1.2/lib/active_record/base.rb:644:in `create' app/models/repository/subversion.rb:59:in `fetch_changesets' app/models/repository/subversion.rb:58:in `reverse_each' app/models/repository/subversion.rb:58:in `fetch_changesets' /var/lib/gems/1.8/gems/activerecord-2.1.2/lib/active_record/connection_adapters/abstract/database_statements.rb:66:in `transaction' /var/lib/gems/1.8/gems/activerecord-2.1.2/lib/active_record/transactions.rb:79:in `transaction' /var/lib/gems/1.8/gems/activerecord-2.1.2/lib/active_record/transactions.rb:98:in `transaction' app/models/repository/subversion.rb:57:in `fetch_changesets' /var/lib/gems/1.8/gems/activerecord-2.1.2/lib/active_record/associations/association_proxy.rb:173:in `send' /var/lib/gems/1.8/gems/activerecord-2.1.2/lib/active_record/associations/association_proxy.rb:173:in `method_missing' app/controllers/repositories_controller.rb:54:in `show' /var/lib/gems/1.8/gems/actionpack-2.1.2/lib/action_controller/base.rb:1166:in `send' /var/lib/gems/1.8/gems/actionpack-2.1.2/lib/action_controller/base.rb:1166:in `perform_action_without_filters' /var/lib/gems/1.8/gems/actionpack-2.1.2/lib/action_controller/filters.rb:579:in `call_filters' /var/lib/gems/1.8/gems/actionpack-2.1.2/lib/action_controller/filters.rb:572:in `perform_action_without_benchmark' /var/lib/gems/1.8/gems/actionpack-2.1.2/lib/action_controller/benchmarking.rb:68:in `perform_action_without_rescue' /usr/lib/ruby/1.8/benchmark.rb:293:in `measure' /var/lib/gems/1.8/gems/actionpack-2.1.2/lib/action_controller/benchmarking.rb:68:in `perform_action_without_rescue' /var/lib/gems/1.8/gems/actionpack-2.1.2/lib/action_controller/rescue.rb:201:in `perform_action_without_caching' /var/lib/gems/1.8/gems/actionpack-2.1.2/lib/action_controller/caching/sql_cache.rb:13:in `perform_action' /var/lib/gems/1.8/gems/activerecord-2.1.2/lib/active_record/connection_adapters/abstract/query_cache.rb:33:in `cache' /var/lib/gems/1.8/gems/activerecord-2.1.2/lib/active_record/query_cache.rb:8:in `cache' /var/lib/gems/1.8/gems/actionpack-2.1.2/lib/action_controller/caching/sql_cache.rb:12:in `perform_action' /var/lib/gems/1.8/gems/actionpack-2.1.2/lib/action_controller/base.rb:529:in `send' /var/lib/gems/1.8/gems/actionpack-2.1.2/lib/action_controller/base.rb:529:in `process_without_filters' /var/lib/gems/1.8/gems/actionpack-2.1.2/lib/action_controller/filters.rb:568:in `process_without_session_management_support' /var/lib/gems/1.8/gems/actionpack-2.1.2/lib/action_controller/session_management.rb:130:in `process' /var/lib/gems/1.8/gems/actionpack-2.1.2/lib/action_controller/base.rb:389:in `process' /var/lib/gems/1.8/gems/actionpack-2.1.2/lib/action_controller/dispatcher.rb:149:in `handle_request' /var/lib/gems/1.8/gems/actionpack-2.1.2/lib/action_controller/dispatcher.rb:107:in `dispatch' /var/lib/gems/1.8/gems/actionpack-2.1.2/lib/action_controller/dispatcher.rb:104:in `synchronize' /var/lib/gems/1.8/gems/actionpack-2.1.2/lib/action_controller/dispatcher.rb:104:in `dispatch' /var/lib/gems/1.8/gems/actionpack-2.1.2/lib/action_controller/dispatcher.rb:120:in `dispatch_cgi' /var/lib/gems/1.8/gems/actionpack-2.1.2/lib/action_controller/dispatcher.rb:35:in `dispatch' /var/lib/gems/1.8/gems/rails-2.1.2/lib/webrick_server.rb:112:in `handle_dispatch' /var/lib/gems/1.8/gems/rails-2.1.2/lib/webrick_server.rb:78:in `service' /usr/lib/ruby/1.8/webrick/httpserver.rb:104:in `service' /usr/lib/ruby/1.8/webrick/httpserver.rb:65:in `run' /usr/lib/ruby/1.8/webrick/server.rb:173:in `start_thread' /usr/lib/ruby/1.8/webrick/server.rb:162:in `start' /usr/lib/ruby/1.8/webrick/server.rb:162:in `start_thread' /usr/lib/ruby/1.8/webrick/server.rb:95:in `start' /usr/lib/ruby/1.8/webrick/server.rb:92:in `each' /usr/lib/ruby/1.8/webrick/server.rb:92:in `start' /usr/lib/ruby/1.8/webrick/server.rb:23:in `start' /usr/lib/ruby/1.8/webrick/server.rb:82:in `start' /var/lib/gems/1.8/gems/rails-2.1.2/lib/webrick_server.rb:62:in `dispatch' /var/lib/gems/1.8/gems/rails-2.1.2/lib/commands/servers/webrick.rb:66 /usr/lib/ruby/1.8/rubygems/custom_require.rb:27:in `gem_original_require' /usr/lib/ruby/1.8/rubygems/custom_require.rb:27:in `require' /var/lib/gems/1.8/gems/activesupport-2.1.2/lib/active_support/dependencies.rb:510:in `require' /var/lib/gems/1.8/gems/activesupport-2.1.2/lib/active_support/dependencies.rb:355:in `new_constants_in' /var/lib/gems/1.8/gems/activesupport-2.1.2/lib/active_support/dependencies.rb:510:in `require' /var/lib/gems/1.8/gems/rails-2.1.2/lib/commands/server.rb:39 /usr/lib/ruby/1.8/rubygems/custom_require.rb:27:in `gem_original_require' /usr/lib/ruby/1.8/rubygems/custom_require.rb:27:in `require' script/server:3 Request Parameters: {"id"=>"app"} Show session dump --- userslist_sort: :key: login :order: asc boardsindex_sort: :key: messages.updated_on :order: desc projectslist_files_sort: :key: attachments.filename :order: asc :user_id: timelogdetails_sort: :key: spent_on :order: desc :query: :project_id: 1 :filters: tracker_id: :values: - "1" :operator: "=" status_id: :values: - "" :operator: o adminprojects_sort: :key: name :order: asc issuesindex_sort: :key: issues.id :order: desc flash: !map:ActionController::Flash::FlashHash {} Response Headers: {"cookie"=>[], "Cache-Control"=>"no-cache"}
(Edit (T. Lecavelier): dump format )
Replies (4)
RE: New project using existing 1.5Gb file:/// svn repository crashes Redmine.
-
Added by Karl Heinz Marbaise almost 16 years ago
Hi,
i would suggest to do the repository scanning for the first time by hand:
Take a look at the FAQ
Kind regards
Karl Heinz Marbase
RE: New project using existing 1.5Gb file:/// svn repository crashes Redmine.
-
Added by Chris Dew almost 16 years ago
Thanks for your reply - I felt a little ashamed at being pointed to the FAQ, but at least the question there was not my question ;-)
Here's the link.
All the best,
Chris.
RE: New project using existing 1.5Gb file:/// svn repository crashes Redmine.
-
Added by Karl Heinz Marbaise almost 16 years ago
Hi Chris,
i wouldn't make you feeling not good sorry...
Thanks for your reply - I felt a little ashamed at being pointed to the FAQ, but at least the question there was not my question ;-)
I think based on this size of your repository it takes simply too long so you should scan your repository by hand before...
Sorry for the wrong line....
Kind regards
Karl Heinz Marbaise
RE: New project using existing 1.5Gb file:/// svn repository crashes Redmine.
-
Added by Fernando Kosh over 15 years ago
Hi Chris and all colleagues,
I have the same problem running Redmine 0.8.2 with a repository containing 2000+ commits.
I follow these instructions:
http://www.redmine.org/wiki/redmine/FAQ#Commits-dont-show-up-in-the-activity-until-I-click-on-Repository
And got a the error again. But, looking into MySQL, I see parts of data from repository. So, I repeat the proccess again and again, always having errors, and every time I see more data in data base. After repeat this step many times, it finally ends fecthing all data and works better now.
It solv my problem, but I want to know if is there other way to do the same.