Redmine

• File contents fail

should be:

• File contents fail to be displayed when the work dir has changed from the last revision (#3421)

Looks good. I'll definitely be giving this a try.

I tested this patch.
Branch has no problem,
but tag lost '.' and '-'.

For example, "tag-111.222" becomes "tag".

Toshi Maruyama wrote:

I tested this patch.
Branch has no problem,
but tag lost '.' and '-'.

For example, "tag-111.222" becomes "tag".

Should be an easy fix, do you know the valid characters for tags off-hand, or should I look them up?

Peter Fern wrote:

Should be an easy fix, do you know the valid characters for tags off-hand, or should I look them up?

Actually, never mind, I'll just admit anything that's not whitespace. Patch on it's way tomorrow.

Peter Fern wrote:

Toshi Maruyama wrote:

I tested this patch.
Branch has no problem,
but tag lost '.' and '-'.

For example, "tag-111.222" becomes "tag".

Should be an easy fix, do you know the valid characters for tags off-hand, or should I look them up?

http://mercurial.selenic.com/wiki/Tag

It can contain any characters except ":" (colon),
"\r" (Carriage Return) or "\n" (Line Feed).

And I test.

$ hg branches
branch_ aaaa bbb                  31:61f3b4c8e7d0

$ hg tags
tip                               31:61f3b4c8e7d0
aaaa aaaa                         29:44823ece1572

Task redmine:fetch_changesets fails.

rake aborted!
undefined method `scmid' for nil:NilClass
r:/Ruby/lib/ruby/gems/1.8/gems/activesupport-2.3.5/lib/active_support/whiny_nil.rb:52:in `method_missing'
c:/redmine/app/models/repository/mercurial.rb:48:in `fetch_changesets'
r:/Ruby/lib/ruby/gems/1.8/gems/activesupport-2.3.5/lib/active_support/core_ext/symbol.rb:11:in `__send__'
r:/Ruby/lib/ruby/gems/1.8/gems/activesupport-2.3.5/lib/active_support/core_ext/symbol.rb:11:in `to_proc'
c:/redmine/app/models/repository.rb:166:in `each'
c:/redmine/app/models/repository.rb:166:in `fetch_changesets'
c:/redmine/lib/tasks/fetch_changesets.rake:22
r:/Ruby/lib/ruby/gems/1.8/gems/rake-0.8.7/lib/rake.rb:636:in `call'
r:/Ruby/lib/ruby/gems/1.8/gems/rake-0.8.7/lib/rake.rb:636:in `execute'
r:/Ruby/lib/ruby/gems/1.8/gems/rake-0.8.7/lib/rake.rb:631:in `each'
r:/Ruby/lib/ruby/gems/1.8/gems/rake-0.8.7/lib/rake.rb:631:in `execute'
r:/Ruby/lib/ruby/gems/1.8/gems/rake-0.8.7/lib/rake.rb:597:in `invoke_with_call_chain'
r:/Ruby/lib/ruby/1.8/monitor.rb:242:in `synchronize'
r:/Ruby/lib/ruby/gems/1.8/gems/rake-0.8.7/lib/rake.rb:590:in `invoke_with_call_chain'
r:/Ruby/lib/ruby/gems/1.8/gems/rake-0.8.7/lib/rake.rb:583:in `invoke'
r:/Ruby/lib/ruby/gems/1.8/gems/rake-0.8.7/lib/rake.rb:2051:in `invoke_task'
r:/Ruby/lib/ruby/gems/1.8/gems/rake-0.8.7/lib/rake.rb:2029:in `top_level'
r:/Ruby/lib/ruby/gems/1.8/gems/rake-0.8.7/lib/rake.rb:2029:in `each'
r:/Ruby/lib/ruby/gems/1.8/gems/rake-0.8.7/lib/rake.rb:2029:in `top_level'
r:/Ruby/lib/ruby/gems/1.8/gems/rake-0.8.7/lib/rake.rb:2068:in `standard_exception_handling'
r:/Ruby/lib/ruby/gems/1.8/gems/rake-0.8.7/lib/rake.rb:2023:in `top_level'
r:/Ruby/lib/ruby/gems/1.8/gems/rake-0.8.7/lib/rake.rb:2001:in `run'
r:/Ruby/lib/ruby/gems/1.8/gems/rake-0.8.7/lib/rake.rb:2068:in `standard_exception_handling'
r:/Ruby/lib/ruby/gems/1.8/gems/rake-0.8.7/lib/rake.rb:1998:in `run'
r:/Ruby/lib/ruby/gems/1.8/gems/rake-0.8.7/bin/rake:31
r:/Ruby/bin/rake:19:in `load'
r:/Ruby/bin/rake:19
(in c:/redmine)
** Invoke redmine:fetch_changesets (first_time)
** Invoke environment (first_time)
** Execute environment
** Execute redmine:fetch_changesets

Sorry, been rather busy this past week. This error does not occur for me, however I'll resync with trunk and try again in the next few days.

In log/test.log.

Shelling out: hg --debug --encoding utf8
 -R "c:\test\test00/" --cwd "c:\test\test00/" log
 --style "c:/redmine/lib/redmine/scm/adapters/mercurial
 /hg-template-1.0-lite.tmpl" -r "f5ead3bc1788":"0" --limit 1
uninitialized constant REXML::Document

So, I added require 'rexml/document' in lib/redmine/scm/adapters/mercurial_adapter.rb.
Then, Task redmine:fetch_changesets finished successfully.

This patch uses wc command. So there is a same probrem with #3836.

Toshi Maruyama wrote:

This patch uses wc command. So there is a same probrem with #3836.

Toshi, thanks for all your testing. I based parts of this on the existing git support, hence the duplicate bug. Using 'wc' was a poor choice, however the same fix will work here. I'll update to trunk and incorporate these changes over the weekend, and check for any other *nix specific shell execution.

Because I am Japanese, I have made a big typo "probrem".
"problem" is correct.

MSysGit has wc.exe in "C:\Program Files\Git\bin".
But TortoiseHg does not have wc.exe in "C:\Program Files\TortoiseHg".

I think migration is impossible because "revision number" and "commit id" are written directly in journals table and wiki_contents table.
Should this feature separate from existing Mercurial adapter and add as new adapter?

I think #3169 "Multiple repositories for projects" is related.

Test units source:trunk/test/unit/mercurial_adapter_test.rb and source:trunk/test/unit/repository_mercurial_test.rb are incompatible.
What should I do?

Toshi Maruyama wrote:

This patch is great. But, new problem occurs.
As described at #3567, Mercurial allows to commit a changeset for earlier date.
But redmine sorts revisions by "committed_on".

This is an existing bug, and honestly, I'm not sure what to do about it. It is particularly difficult to deal with since with this patch, we do not store the incremental revision numbers in the database at all. I'm open to suggestions on how to resolve this, but I think for now I will not deal with it until everything else is signed off.

Toshi Maruyama wrote:

I think migration is impossible because "revision number" and "commit id" are written directly in journals table and wiki_contents table.
Should this feature separate from existing Mercurial adapter and add as new adapter?

I don't like the idea of creating a separate adapter, however I think that the journals and wiki_contents references could be updated with a db migration? That was my original plan. I will see if I can get Eric or one of the other devs to make some comments on this once the other issues are resolved.

Toshi Maruyama wrote:

Test units source:trunk/test/unit/mercurial_adapter_test.rb and source:trunk/test/unit/repository_mercurial_test.rb are incompatible.
What should I do?

I'll work through all the tests soon (hopefully next week), and update them to test all the new functionality, as well as to work with the new id format.

Peter Fern wrote:

I don't like the idea of creating a separate adapter, however I think that the journals and wiki_contents references could be updated with a db migration? That was my original plan. I will see if I can get Eric or one of the other devs to make some comments on this once the other issues are resolved.

Just let me know once you are ready. I'm watching this issue now but I've only used Mercurial once (I use git mostly).

Peter Fern wrote:

I'll work through all the tests soon (hopefully next week), and update them to test all the new functionality, as well as to work with the new id format.

Peter, I found your git repository on github.
Can I use this?

I update source:trunk/test/unit/repository_mercurial_test.rb.
And I create git repository on github.
http://github.com/marutosi/redmine/commits/marutosi

Toshi Maruyama wrote:

I update source:trunk/test/unit/repository_mercurial_test.rb.
And I create git repository on github.
http://github.com/marutosi/redmine/commits/marutosi

Toshi, sorry for the lack of updates here - I wrote a large reply some time ago, but it looks like it didn't post correctly. It detailed the changes I had pushed to my Github fork, it's location, and my strategy for moving forward.

I started to look at the tests over the weekend, but the current test repository does not cover all the features that the new code does. My idea was to base a new Mercurial test suite on the Git tests (though they also seem a little inadequate for testing all the Git functionality), but the repository would need recreation from scratch as simply converting the current Git repository will not work due to the differences in branching with Mercurial vs Git. If you want to move forward on updating the Mercurial tests, this is the route I would suggest.

As to correct sorting, the only way I can see of doing it would be to add a field `scmsort` and store the revision numbers there. This should work with some other minor modifications, but as it requires a schema change, I'm a little reluctant to implement it that way... thoughts?

I created wiki converter rake task.
http://github.com/marutosi/redmine/commit/ec8d2de04876f2a77aeb2ca88f4407c0e1db8b21

Wikis have revisions on DB, so old revisions are not changed and converted text are saved as new revision.

I found the reason of #3449 "Redmine Takes Too Long" everytime.
The reason is "db_revision" and "scm_revision" are not equal everytime at Repository::Mercurial.fetch_changesets.

• "db_revision" is latest changeset because this is sorted by "committed_on".
• "scm_revision" is tip and tip is not latest changeset.

There are two case.

1. Child changeset has earlier date than parent.
   • Commit with --date option
   • MQ (Mercurial Queues) http://mercurial.selenic.com/wiki/MqExtension
2. Multiple Heads. http://mercurial.selenic.com/wiki/MultipleHeads
   • Named branches. http://mercurial.selenic.com/wiki/NamedBranches

This condition occurs frequently.

I tried to resolve this problem referring patch of #3449.

class Repository::Mercurial < Repository
.
.
.
  def latest_changeset
    @latest_changeset ||= changesets.find (
         :first ,
         :order => "LPAD(revision,10,'0000000000') DESC" 
      )
  end

But, SQLite does not support "LPAD".

Adding a field at "changesets" table is only way to resolve this performance problem.
So I agree to add "scmsort" field, but I think "scm_number" or "scm_no" or "hg_revno" is better.

Additionally, git rabase changeset has earlier date than parent.

Toshi Maruyama wrote:

I tried to resolve this problem referring patch of #3449.

I mistake issue number. Correct is #3567 - "Sorting for changesets might go wrong on Mercurial repos".

I reconsidered above performance problem.
Mercurial revision number is sequential from 0.
So, "changesets.count - 1" is latest Mercurial revision number in DB.

I create a patch and attach at #3449 - Redmine Takes Too Long On Large Mercurial Repository.

Toshi Maruyama wrote:

Adding a field at "changesets" table is only way to resolve this performance problem.
So I agree to add "scmsort" field, but I think "scm_number" or "scm_no" or "hg_revno" is better.

Additionally, git rabase changeset has earlier date than parent.

The reason I suggested scmsort (maybe scm_order) was so that it could be used for other SCMs that require an order other than committed_on - in Mercurial that would be the revision number, but in some other SCM, you might use a unix timestamp or some such...

Toshi Maruyama wrote:

I reconsidered above performance problem.
Mercurial revision number is sequential from 0.
So, "changesets.count - 1" is latest Mercurial revision number in DB.

I create a patch and attach at #3449 - Redmine Takes Too Long On Large Mercurial Repository.

Be careful - with history editing, the current code does not clear missing revisions, so your count will not reflect the actual number of commits, or the current revision number, in the repository.

Peter Fern wrote:

Be careful - with history editing, the current code does not clear missing revisions, so your count will not reflect the actual number of commits, or the current revision number, in the repository.

Thanks. I test "hg strip". Changesets ramain on DB.

History editing is rare case on shared repository. I think everytime check of histoy editing is high cost. For this case, should Redmine provide another form?

Toshi Maruyama wrote:

History editing is rare case on shared repository. I think everytime check of histoy editing is high cost. For this case, should Redmine provide another form?

This overhaul already takes care of it - if the tip id and number of changesets in the repository and database match, no sync is done, otherwise we match off scmids against the log and prune or add as necessary. It's not that heavy really, and it's necessary for consistency... Git does the same too.

Peter Fern wrote:

Toshi Maruyama wrote:

History editing is rare case on shared repository. I think everytime check of histoy editing is high cost. For this case, should Redmine provide another form?

This overhaul already takes care of it - if the tip id and number of changesets in the repository and database match, no sync is done, otherwise we match off scmids against the log and prune or add as necessary. It's not that heavy really, and it's necessary for consistency... Git does the same too.

I test Mercurial crew repository.
http://hg.intevation.org/mercurial/crew

Completed in 177313ms (View: 777, DB: 5245) | 200 OK [http://localhost/projects/hgcrew/repository]
Completed in 175350ms (View: 792, DB: 5132) | 200 OK [http://localhost/projects/hgcrew/repository]
Completed in 175149ms (View: 792, DB: 5136) | 200 OK [http://localhost/projects/hgcrew/repository]

$ du -sh .hg
25M     .hg
$ du -sh .
41M     .
$ find  .hg -type f | wc
   1488    1488   63468
$ find  . -type f | wc
   3038    3038  120048

$ hg log  -l2
changeset:   10425:4cfd0d56be6d
tag:         tip
user:        XXXXXXXXXXXXXXXXXXXX
date:        Thu Feb 11 21:11:59 2010 +0100
summary:     doc: add missing documentation for http_proxy.always

changeset:   10424:677f15da38c1
user:        YYYYYYYYYYYYYYYYYYYY
date:        Thu Feb 11 20:42:20 2010 +0100
summary:     url: proxy handling, simplify and correctly deal with IPv6

Toshi Maruyama wrote:

I test Mercurial crew repository.
http://hg.intevation.org/mercurial/crew

Most of this time is spent fetching details for individual metadata for items (parallelizing the metadata retrieval might be a good idea to improve display time):

Completed in 13612ms (View: 661, DB: 54) | 200 OK [http://localhost/projects/hgcrew/repository]
Completed in 14331ms (View: 601, DB: 21) | 200 OK [http://localhost/projects/hgcrew/repository]
Completed in 13540ms (View: 605, DB: 23) | 200 OK [http://localhost/projects/hgcrew/repository]

Check your error log - I had some problems importing that repository the first time, since when I created the database I did not have the default charset set to UTF-8, and some commit logs contain non-latin characters. I was seeing this:

  Changeset Load Including Associations (0.0ms)   Mysql::Error: Illegal mix of collations (latin1_swedish_ci,IMPLICIT) and (utf8_general_ci,COERCIBLE) for operation '=': SELECT `changesets`.`id` AS t0_r0, `changesets`.`repository_id` AS t0_r1, `changesets`.`revision` AS t0_r2, `changesets`.`committer` AS t0_r3, `changesets`.`committed_on` AS t0_r4, `changesets`.`comments` AS t0_r5, `changesets`.`commit_date` AS t0_r6, `changesets`.`scmid` AS t0_r7, `changesets`.`user_id` AS t0_r8, `users`.`id` AS t1_r0, `users`.`login` AS t1_r1, `users`.`hashed_password` AS t1_r2, `users`.`firstname` AS t1_r3, `users`.`lastname` AS t1_r4, `users`.`mail` AS t1_r5, `users`.`mail_notification` AS t1_r6, `users`.`admin` AS t1_r7, `users`.`status` AS t1_r8, `users`.`last_login_on` AS t1_r9, `users`.`language` AS t1_r10, `users`.`auth_source_id` AS t1_r11, `users`.`created_on` AS t1_r12, `users`.`updated_on` AS t1_r13, `users`.`type` AS t1_r14, `users`.`identity_url` AS t1_r15 FROM `changesets` LEFT OUTER JOIN `users` ON `users`.id = `changesets`.user_id AND (`users`.`type` = 'User' OR `users`.`type` = 'AnonymousUser' ) WHERE (`changesets`.repository_id = 1 AND (`changesets`.`committer` = 'Wojciech Miłkowski <user@domain.tld>')) ORDER BY changesets.committed_on DESC, changesets.id DESC LIMIT 1
  SQL (0.1ms)   ROLLBACK

I fixed it by converting all tables to UTF-8:

echo 'show tables' |mysql -u<redminedbuser> -p<redminedbpass> <redminedbname> |grep -v Tables_in_redmine_dev_mercurial |xargs -i{} echo 'ALTER TABLE {} CONVERT TO CHARACTER SET utf8 COLLATE utf8_general_ci;' |mysql --u<redminedbuser> -p<redminedbpass> <redminedbname>

I should probably commit the changesets in batches, rather than all at once since we could use a lot of memory, and an error causes all the changesets to roll back, which hurts on initial import...

Fix should be:

echo 'show tables' |mysql -u<redminedbuser> -p<redminedbpass> <redminedbname> |grep -v Tables_in_<redminedbname> |xargs -i{} echo 'ALTER TABLE {} CONVERT TO CHARACTER SET utf8 COLLATE utf8_general_ci;' |mysql --u<redminedbuser> -p<redminedbpass> <redminedbname>

Where you replace the parameters <redminedbuser>, <redminedbpass>, and <redminedbname>..

I use native windows ruby (not cygwin) and SQLite.

Toshi Maruyama wrote:

I use native windows ruby (not cygwin) and SQLite.

Does SQLite deal with UTF-8? Do you have any errors in your logs?

There won't be many (any?) production installs running on SQLite, but I can't believe there could be an order of magnitude difference in performance for this operation, and I've confirmed that no update occurs when the revision count matches in DB and repository, and the tip IDs match. There might be a performance gain on merging changes by moving from 'NOT IN' to 'IN' when deleting, but I'd have to test, and it can't account for what you're seeing.

I would suggest that you must be encountering an error importing the repository, causing Redmine to attempt to re-import each time... what is changelogs.count for that repository?

Peter Fern wrote:

what is changelogs.count for that repository?

Err, I mean changesets.count

SELECT count(*) AS count_all FROM "changesets" WHERE ("changesets".repository_id = 15)
  => 10426

  def fetch_changesets
    scm_info = scm.info
    return unless scm_info or scm_info.lastrev.nil?

    db_revision = latest_changeset ? latest_changeset.scmid.to_s : 0
    scm_revision = scm_info.lastrev.scmid.to_s
    # Save ourselves an expensive operation if we're already up to date

    scm_revcount = scm.num_revisions
    db_revcount = changesets.count

    logger.debug( "scm_revcount :" + scm_revcount.to_s )
    logger.debug( "db_revcount  :" + db_revcount.to_s )

Shelling out: hg -R "R:\WEB-DOWN\hg-crew/" log -r :tip --template='
'
  [4;35;1mSQL (0.0ms)[0m   [0mSELECT count(*) AS count_all FROM "changesets" WHERE ("changesets".repository_id = 15) [0m
scm_revcount :10427
db_revcount  :10426
Shelling out: hg -R "R:\WEB-DOWN\hg-crew/" log -r :tip --template='
'
  [4;36;1mCACHE (0.0ms)[0m   [0;1mSELECT count(*) AS count_all FROM "changesets" WHERE ("changesets".repository_id = 15) [0m
Shelling out: hg --debug --encoding utf8 -R "R:\WEB-DOWN\hg-crew/" --cwd "R:\WEB-DOWN\hg-crew/" log --style "S:/redmine-00/lib/redmine/scm/adapters/mercurial/hg-template-1.0-lite.tmpl" -r :"4cfd0d56be6dcbcfac90bf95e8f8750ab66fd9aa" 
  [4;35;1mChangeset Load (3369.6ms)[0m   [0mSELECT * FROM "changesets" WHERE ("changesets".repository_id = 15) ORDER BY changesets.committed_on DESC, changesets.id DESC[0m
  [4;36;1mChangeset Delete all (249.6ms)[0m   [0;1mDELETE FROM "changesets" WHERE (scmid NOT IN ('9117c6561b0bd7792fa13b50d28239d51b78e51f',

Hmm, so, yours is off-by-one... I wonder if it's a single commit that is not being inserted into the database correctly? You should get an error... can you upload your log as an attachment here?

Mine syncs fine on MySQL (with UTF-8):

DB rev: 1c50a954a5244a8da8d6d2311dd12d6a5f6df357, SCM rev: 1c50a954a5244a8da8d6d2311dd12d6a5f6df357

  SQL (1.1ms)   SELECT count(*) AS count_all FROM `changesets` WHERE (`changesets`.repository_id = 1) 
Shelling out: hg -R '/home/user/workspace/crew/' log -r :tip --template='
'

DB count: 10430, SCM count: 10430

  CACHE (0.0ms)   SELECT count(*) AS count_all FROM `changesets` WHERE (`changesets`.repository_id = 1) 
Shelling out: hg -R '/home/user/workspace/crew/' log -r :tip --template='

On MSYS/MinGW .

$ hg -R "R:\WEB-DOWN\hg-crew/" log -r :tip --template='\n' | wc
  10426       0   10426

$ which wc
/bin/wc.exe

Are you using the following code in lib/redmine/scm/adapters/mercurial_adapter.rb?

        def num_revisions
          cmd = "#{HG_BIN} -R #{target('')} log -r :tip --template='\n'" 
          shellout(cmd) {|io| io.readlines.size}
        end

Peter Fern wrote:

Are you using the following code in lib/redmine/scm/adapters/mercurial_adapter.rb?
[...]

Yes.

Toshi Maruyama wrote:

Peter Fern wrote:

Are you using the following code in lib/redmine/scm/adapters/mercurial_adapter.rb?
[...]

Yes.

http://github.com/marutosi/redmine/blob/marutosi/lib/redmine/scm/adapters/mercurial_adapter.rb#L150

Well... that's odd - for some reason io.readlines.size is one larger for you on Windows, perhaps there is an extra line of output generated somehow. I'm not quite sure how to resolve that since I can't reproduce it.

        def num_revisions
          # cmd = "#{HG_BIN} -R #{target('')} log -r :tip --template='\n'" 
          # shellout(cmd) {|io| io.readlines.size}
          cmd = "#{HG_BIN} -R #{target('')} log -r :tip --template='{rev}\n'" 
          i = 0
          logger.debug ( "test start"  )
          shellout(cmd) do |io|
            io.each_line do |line|
              i += 1
              logger.debug ( "test:" + line )
            end
          end
          logger.debug ( "test finish i: " + i.to_s  )
        end

test start
Shelling out: hg -R "R:\WEB-DOWN\REDMINE\hg-test-repo\redmine-test-mercurial-repository/" log -r :tip --template='{rev}
'
test:'0
test:''1
test:''2
test:''3
test:''4
test:''5
test:'
test finish i: 7
  [4;36;1mCACHE (0.0ms)[0m   [0;1mSELECT count(*) AS count_all FROM "changesets" WHERE ("changesets".repository_id = 19) [0m

Peter Fern wrote:

Hmm, so you have a trailing line of output... try the attached patch.

Actually, that won't work. I just noticed all the "'" quotes, so something about the shell there is interpreting the quotes literally. Give me a moment.

Actually, should probably just shell_quote it

        def num_revisions
          # cmd = "#{HG_BIN} -R #{target('')} log -r :tip --template='\n'" 
          # shellout(cmd) {|io| io.readlines.size}
          # cmd = "#{HG_BIN} -R #{target('')} log -r :tip --template='{rev}\n'" 
          cmd = "#{HG_BIN} -R #{target('')} log -r :tip --template=\"{rev}\n\"" 
          i = 0
          logger.debug ( "test start"  )
          shellout(cmd) do |io|
            io.each_line do |line|
              i += 1
              logger.debug ( "test:" + line )
            end
          end
          logger.debug ( "test finish i: " + i.to_s  )
          i
        end

Shelling out: hg -R "R:\WEB-DOWN\REDMINE\hg-test-repo\redmine-test-mercurial-repository/" log -r :tip --template="{rev}
" 
test:0
test:1
test:2
test:3
test:4
test:5
test finish i: 6
  [4;35;1mCACHE (0.0ms)[0m   [0mSELECT count(*) AS count_all FROM "changesets" WHERE ("changesets".repository_id = 19) [0m

http://github.com/pdf/redmine/commit/4df6b436d8a7dd0a1df242aa16e1fb33ff5618dc

        def num_revisions
          # cmd = "#{HG_BIN} -R #{target('')} log -r :tip --template='\n'" 
          # shellout(cmd) {|io| io.readlines.size}
          # cmd = "#{HG_BIN} -R #{target('')} log -r :tip --template='{rev}\n'" 
          cmd = "#{HG_BIN} -R #{target('')} log -r :tip --template=#{shell_quote('{rev}\n')}" 
          i = 0
          logger.debug ( "test start"  )
          shellout(cmd) do |io|
            io.each_line do |line|
              i += 1
              logger.debug ( "test:" + line )
            end
          end
          logger.debug ( "test finish i: " + i.to_s  )
          i
        end

test start
Shelling out: hg -R "R:\WEB-DOWN\REDMINE\hg-test-repo\redmine-test-mercurial-repository/" log -r :tip --template="{rev}\n" 
test:0
test:1
test:2
test:3
test:4
test:5
test finish i: 6

Alright, that should improve your load times significantly. There are probably still some other optimizations to be done, but I believe it's important to first be accurate, then be fast.

Git has this single quotes problem too (#3836 - Git repository support fails on Windows).

source:/tags/0.9.2/lib/redmine/scm/adapters/git_adapter.rb#L113

So, I tried to fix it.
But, "GitAdapter#num_revisions" method removed from trunk by r3394 and logic changed.

Peter Fern wrote:

Alright, that should improve your load times significantly. There are probably still some other optimizations to be done, but I believe it's important to first be accurate, then be fast.

New Repository::Git#fetch_changesets is good for performance.
Mercurial has sequential revision number, so this process is easy and fast.

1. From database get revision number and hash id ordered by revision number from tip.
2. If revision number and hash id are not equal, remove changeset from database.
3. If revision number and hash id are equal, loop finish.
4. Read new changesets from repository and insert into database.

I finished coding and unit test with DB migrate of adding "scm_order" field.
http://github.com/marutosi/redmine/commits/c72615fac2f99be855a3189e28fa78ed15ad2c96

Functional test "test_directory_entry" of repositories_mercurial_controller_test.rb fails.
I remove "MercurialAdapter#entry(path=nil, identifier=nil)" method, then this test finished with no error.

I finished coding, unit test and functional test.
http://github.com/marutosi/redmine/commits/hg-overhaul
http://github.com/marutosi/redmine/commit/8b90fe01b1cf0de746327fd00937e60d9390b753

I got a feedback on github and fixed some bugs.
I create new tag "hg-overhaul-0.9.3-20100310" for Redmine 0.9.3
http://github.com/marutosi/redmine/tree/hg-overhaul-0.9.3-20100310

You can download tar ball or zip at following link.
http://github.com/marutosi/redmine/tarball/hg-overhaul-0.9.3-20100310
http://github.com/marutosi/redmine/zipball/hg-overhaul-0.9.3-20100310

Toshi Maruyama wrote:

I got a feedback on github and fixed some bugs.
I create new tag "hg-overhaul-0.9.3-20100310" for Redmine 0.9.3

Executing ruby script/runner "Repository.fetch_changesets" -e production no longer work with these patches.
Issue tracker where not updated if you don't visit manually the repository page.

By the way you did an excellent work ;)

Alessio Caiazza wrote:

Toshi Maruyama wrote:

I got a feedback on github and fixed some bugs.
I create new tag "hg-overhaul-0.9.3-20100310" for Redmine 0.9.3

Executing ruby script/runner "Repository.fetch_changesets" -e production no longer work with these patches.
Issue tracker where not updated if you don't visit manually the repository page.

By the way you did an excellent work ;)

Thank you for your feedback. Can you use git? Please pull and use latest revision of "hg-overhaul-0.9" branch.

Toshi Maruyama wrote:

Thank you for your feedback. Can you use git? Please pull and use latest revision of "hg-overhaul-0.9" branch.

git? :(
Yes the "hg-overhaul-0.9" branch fixed the problem.

Today I made a patch, based on this branch, for editing some options of .hg/hgrc inside project repository settings.

This patch needs some more work, but I'll publish it very soon (I hope).

toshio harita - I apologise for my lack of attention to this issue, I've been out of town for most of the last 6 weeks or so, but it's excellent to see that you've been able continue with this in my absence. I will try and set aside some time in the next few days for some review and testing.

Alessio Caiazza wrote:

Toshi Maruyama wrote:

I got a feedback on github and fixed some bugs.
I create new tag "hg-overhaul-0.9.3-20100310" for Redmine 0.9.3

Executing ruby script/runner "Repository.fetch_changesets" -e production no longer work with these patches.
Issue tracker where not updated if you don't visit manually the repository page.

By the way you did an excellent work ;)

I've found an issue, but I'm not sure if it's related to your work or not.
If you change a repo url, the root_url attribute didn't get updated, so executing Repository.fetch_changesets didn't get any updates, but viewing the Repository page updates all the issues.

Alessio Caiazza wrote:

I've found an issue, but I'm not sure if it's related to your work or not.
If you change a repo url, the root_url attribute didn't get updated, so executing Repository.fetch_changesets didn't get any updates, but viewing the Repository page updates all the issues.

Thanks. I change repo url editable.
http://github.com/marutosi/redmine/blob/d54a7f267a2f48112e268751c4347a0476b0cea2/app/helpers/repositories_helper.rb#L169
I start an investigation.

for what it is worth, mercurial 1.5 has been released last week and - finally - can output log in xml :

http://mercurial.selenic.com/wiki/WhatsNew

Alessio Caiazza wrote:

I've found an issue, but I'm not sure if it's related to your work or not.
If you change a repo url, the root_url attribute didn't get updated, so executing Repository.fetch_changesets didn't get any updates, but viewing the Repository page updates all the issues.

I fixed and pushed.
http://github.com/marutosi/redmine/tree/hg-overhaul-0.9
http://github.com/marutosi/redmine/commit/2f69eb14648bb3f3da773df407186922fd55c02d
Can you try this?

Toshi Maruyama wrote:

Alessio Caiazza wrote:

I've found an issue, but I'm not sure if it's related to your work or not.
If you change a repo url, the root_url attribute didn't get updated, so executing Repository.fetch_changesets didn't get any updates, but viewing the Repository page updates all the issues.

I fixed and pushed.
http://github.com/marutosi/redmine/tree/hg-overhaul-0.9
http://github.com/marutosi/redmine/commit/2f69eb14648bb3f3da773df407186922fd55c02d
Can you try this?

cd /var/www/nolith/rails/redmine && ruby script/runner "Repository.fetch_changesets" -e production
/var/www/nolith/rails/redmine-hg/vendor/rails/railties/lib/commands/runner.rb:48: undefined method `[]' for nil:NilClass (NoMethodError)
        from /var/www/nolith/rails/redmine-hg/lib/redmine/scm/adapters/mercurial_adapter.rb:47:in `hgversion'
        from /var/www/nolith/rails/redmine-hg/lib/redmine/scm/adapters/mercurial_adapter.rb:37:in `client_version'
        from /var/www/nolith/rails/redmine-hg/lib/redmine/scm/adapters/mercurial_adapter.rb:62:in `lite_template_path'
        from /var/www/nolith/rails/redmine-hg/lib/redmine/scm/adapters/mercurial_adapter.rb:261:in `revisions'
        from /var/www/nolith/rails/redmine-hg/lib/redmine/scm/adapters/mercurial_adapter.rb:155:in `lastrev'
        from /var/www/nolith/rails/redmine-hg/app/models/repository/mercurial.rb:124:in `fetch_changesets'
        from /var/www/nolith/rails/redmine-hg/vendor/rails/activerecord/lib/active_record/associations/association_proxy.rb:217:in `send'
        from /var/www/nolith/rails/redmine-hg/vendor/rails/activerecord/lib/active_record/associations/association_proxy.rb:217:in `method_missing'
        from /var/www/nolith/rails/redmine-hg/app/models/repository.rb:183:in `fetch_changesets'
        from /var/www/nolith/rails/redmine-hg/app/models/repository.rb:181:in `each'
        from /var/www/nolith/rails/redmine-hg/app/models/repository.rb:181:in `fetch_changesets'
        from (eval):1
        from /usr/local/lib/site_ruby/1.8/rubygems/custom_require.rb:31:in `eval'
        from /var/www/nolith/rails/redmine-hg/vendor/rails/railties/lib/commands/runner.rb:48
        from /usr/local/lib/site_ruby/1.8/rubygems/custom_require.rb:31:in `gem_original_require'
        from /usr/local/lib/site_ruby/1.8/rubygems/custom_require.rb:31:in `require'
        from script/runner:3

Maybe you should set root_url = url

Alessio Caiazza wrote:

Toshi Maruyama wrote:

Alessio Caiazza wrote:

I've found an issue, but I'm not sure if it's related to your work or not.
If you change a repo url, the root_url attribute didn't get updated, so executing Repository.fetch_changesets didn't get any updates, but viewing the Repository page updates all the issues.

I fixed and pushed.
http://github.com/marutosi/redmine/tree/hg-overhaul-0.9
http://github.com/marutosi/redmine/commit/2f69eb14648bb3f3da773df407186922fd55c02d
Can you try this?

[...]

Maybe you should set root_url = url

I fixed error handling and pushed github.
Your trace means version check error.
Do you upgrade Mercurial and hg in PATH?
Or, "hg --version" does not output "version"?

On Fedora 12.

$ LANG=it_IT.UTF-8 hg --version
Mercurial Distributed SCM (version 1.5)

$ LANG=ja_JP.UTF-8 hg --version
Mercurial Distributed SCM (version 1.5)

$ rpm -q mercurial
mercurial-1.5-2.fc12.i686

Alessio Caiazza wrote:

Maybe you should set root_url = url

root_url set at
source:tags/0.9.3/lib/redmine/scm/adapters/abstract_adapter.rb#L50 .

Toshi Maruyama wrote:

Alessio Caiazza wrote:

Toshi Maruyama wrote:

Alessio Caiazza wrote:

I've found an issue, but I'm not sure if it's related to your work or not.
If you change a repo url, the root_url attribute didn't get updated, so executing Repository.fetch_changesets didn't get any updates, but viewing the Repository page updates all the issues.

I fixed and pushed.
http://github.com/marutosi/redmine/tree/hg-overhaul-0.9
http://github.com/marutosi/redmine/commit/2f69eb14648bb3f3da773df407186922fd55c02d
Can you try this?

[...]

Maybe you should set root_url = url

I fixed error handling and pushed github.
Your trace means version check error.

I'm sorry. I didn't checked the error, only cut and pasted it. :(

Do you upgrade Mercurial and hg in PATH?

That's strage, I didn't....

Or, "hg --version" does not output "version"?
[...]

 $ locale | grep LANG
LANG=it_IT.UTF-8

 $ hg --version
Mercurial SCM Distribuito (versione 1.4.3)

Copyright (C) 2005-2010 Matt Mackall <mpm@selenic.com> e altri
Questo è software libero; vedere i sorgenti per le condizioni di copia.
Non c'è alcuna garanzia; neppure di COMMERCIABILITÀ o IDONEITÀ AD UNO
SCOPO PARTICOLARE.

Italian localization uses "versione" instead of "version"

By the way I pulled your commits and now it works.
Thanks a lot.

I'll really appreciate an updated patch.
An svn patch, built against trunk, would be easy to deploy when redmine installation is a svn checkout.

Thanks!

Ale Franz wrote:

I'll really appreciate an updated patch.
An svn patch, built against trunk, would be easy to deploy when redmine installation is a svn checkout.

Thanks!

Thanks. This overhaul includes DB migrate. So, patch for SVN trunk is difficult.
I created git branch for SVN trunk.
http://github.com/marutosi/redmine/tree/hg-overhaul-svn-trunk
But, now main branch is hg-overhaul-0.9
http://github.com/marutosi/redmine/tree/hg-overhaul-0.9

OK. I checked out branch hg-overhaul-0.9
In the repository tree, I don't have information (date, author, ..) about the files/folder in the root.
btw, it works fine in subfolders.
any idea? I have already tried to remove and add again the repository in redmine

thanks!

Ale Franz wrote:

OK. I checked out branch hg-overhaul-0.9
In the repository tree, I don't have information (date, author, ..) about the files/folder in the root.
btw, it works fine in subfolders.
any idea? I have already tried to remove and add again the repository in redmine

thanks!

I start to write announce, now this include Japanese.
http://bitbucket.org/marutosi/redmine-test-mercurial-repository/src/cda5c85f3ee7/announce.hg-overhaul.txt
I will describe details later.

I hardcode limiter of "hg log -l1 FILE".
because this is very heavy.

• http://www.redmine.org/issues/4773#note-13
• #5096

You can change limiter "@@limit_include_file_revs" by hand.
http://github.com/marutosi/redmine/blob/hg-overhaul-0.9/lib/redmine/scm/adapters/mercurial_adapter.rb#L32
This limiter is number of files per directory.

Mercurial branch is "named branch".
So, if named brance stored in DB, I think browsing is more fast.

Toshi Maruyama wrote:

Mercurial branch is "named branch".
So, if named brance stored in DB, I think browsing is more fast.

"brance" => branch"

Mercurial branch is "named branch".
So, if named branch stored in DB, I think browsing is more fast.

Thanks Ale. Your suggestion and hg extension are great.
Redmine repository browser require information of "file" or "dir".
source:/tags/0.9.3/lib/redmine/scm/adapters/abstract_adapter.rb#L238
Mercurial can not handle directory.
So, this is difficult.

This overhaul support branches and this code is based on git code.
Git branch is a pointer of head. So, git need to run "git log -n 1".
Mercurial branch is "named branch" and named branch fixed on changesets.
And, I think Entry#lastrev requires last revision of branch.

Redmine 0.9 Repository::Mercurial build lastrev from DB.
source:/tags/0.9.3/app/models/repository/mercurial.rb#L32
But branch information are not stored in DB.
So, Redmine need to run "hg log -l1 --only-branch BRANCH FILE".

First stage of my future plans.

• Tag is a hash of name and shortid, so redmine can use only DB info with shortid of tag.
• If branch is single including closed branch, redmine can use only DB info.
• Branch support can be selectable.
  Personally, I don't want to use named branch aggressively. Because named branch can not delete.

Yuya Nishihara wrote:

1. We need to reduce the number of hg calls. Ale's idea seems great.
   Once packing all hg calls into single command, the performance should be terribly improved.

If you let me know which information you need to retrieve from the repository, I'll update the extension to meet you requirements.
I'm quite sure you need: # filtered output on a per directory basis # what else?

Ale Franz wrote:

If you let me know which information you need to retrieve from the repository, I'll update the extension to meet you requirements.
I'm quite sure you need:

1. filtered output on a per directory basis
2. what else?

Hi, it's really helpful! :)
I'm yet reached that point, hacking around "node id" issue now, but maybe Toshi has some idea?

Yuya Nishihara wrote:

I'm not sure about #3449, so it's not included yet.

I talked with Toshi, and found the patch happened to fix #3449.
The underlaying problem is latest_changeset does not returns the latest revision. Since my patch changes the order, now it should return the correct one.

Any chance to reconsider #3724?

I am not the only one who prefers revision number. I really would like to see this as an option.
http://www.redmine.org/issues/3724#note-14 (or my note-20)

It is very nice, that the patch queue doesn't need db migration, so I can run both versions at same time. Progress is quite awesome in this respect.

One little glitch, the fetch_changesets somehow does ignore the seconds from commit dates, this causes unordered activity lists if the dev commits multiple changesets in same minute. The database already does support seconds, so this should be fixeable, I hope. :-)

I hope for some reposman support like creating of the repo after project creating and choosing mercurial as VCS. I already use Redmine togehter with hgredmine.

Greets and Thanks so far...
-Ammler http://dev.openttdcoop.org (or http://new.dev.openttdcoop.org with this mq)

Toshi Maruyama wrote:

'isodatesec' supported at Mercuiral revision 8999d1249171

It means 'isodatesec' is available since Mercurial 1.0. :)

This patch is for Redmine 0.9-stable.

Thanks, now mercurial backend stores precise committed_on and commit_date.
I imported your patch to my mq-issue4455 repo, but IMHO, it's safe enough to be got in upstream separately.

Marcel Gmür wrote:

I hope for some reposman support like creating of the repo after project creating and choosing mercurial as VCS. I already use Redmine togehter with hgredmine.

I don't know what hgredmine is.
Reposman has already enabled to create repositories and register to Redmine with a following command.

$ ruby extra/svn/reposman.rb -s /tmp/hg-repo  -u /tmp/hg-repo \
    -r localhost:3000 -o foo -g foo --scm mercurial --command "hg init" 

Yuya's patch fails in "Activity" of Subversion project.

NoMethodError in Projects#activity

Showing app/views/projects/activity.rhtml where line #13 raised:

undefined method `truncate_long_revision_name' for RepositoriesHelper:Module

Extracted source (around line #13):

10:     <%= avatar(e.event_author, :size => "24") if e.respond_to?(:event_author) %>
11:   <span class="time"><%= format_time(e.event_datetime, false) %></span>
12:   <%= content_tag('span', h(e.project), :class => 'project') if @project.nil? || @project != e.project %>
13:   <%= link_to format_activity_title(e.event_title), e.event_url %></dt>
14:   <dd><span class="description"><%= format_activity_description(e.event_description) %></span>
15:   <span class="author"><%= e.event_author if e.respond_to?(:event_author) %></span></dd>
16: <% end -%>

/somewhere/app/helpers/repositories_helper.rb:32:in `format_revision'
/somewhere/app/models/changeset.rb:26
/somewhere/app/views/projects/activity.rhtml:13:in `_run_rhtml_app47views47projects47activity46rhtml'
/somewhere/app/views/projects/activity.rhtml:8:in `each'
/somewhere/app/views/projects/activity.rhtml:8:in `_run_rhtml_app47views47projects47activity46rhtml'
/somewhere/app/views/projects/activity.rhtml:5:in `each'
/somewhere/app/views/projects/activity.rhtml:5:in `_run_rhtml_app47views47projects47activity46rhtml'

Toshi Maruyama wrote:

Yuya's patch fails in "Activity" of Subversion project.

Fixed as http://bitbucket.org/yuja/redmine-mq-issue4455/changeset/e5b02f53025a , though it's workaround yet.

Thanks for reporting it.

it also fails on overall activity, rss feed works.

Marcel Gmür wrote:

it also fails on overall activity, rss feed works.

Yes, the previous patch had a problem with non-hg repos, including Subversion.
If it occurs on the latest one from http://bitbucket.org/yuja/redmine-mq-issue4455 , it would be another problem.

Well, I also get an error if I try to fetch the changesets of a hg repo:

/usr/lib64/ruby/gems/1.8/gems/rails-2.3.5/lib/commands/runner.rb:48: #<REXML::ParseException: No close tag for /log> (REXML::ParseException)                                                        

Marcel Gmür wrote:

Well, I also get an error if I try to fetch the changesets of a hg repo:
[...]

Please tell me and Yuya where hg repo is.

I don't know, what you exactly need, here my full output of the fetch changeset run:

http://pastebin.com/FkVeC5N0

The error only happens, if there is something fetch.

I run now both apps parallel, one is the svn vanilla 0.9-stable, the other the qclone from bitbucket.org, My public address is the uses your patch queue, but I still fetch changesets with the svn version, so I don't get seconds, but it works.

Greets
Ammler

Marcel Gmür wrote:

I don't know, what you exactly need, here my full output of the fetch changeset run:

http://pastebin.com/FkVeC5N0

Thanks for testing the patches. Could you give us the Mercurial version?
I eliminated "# HG doesn't close the XML Document..." hack temporarily, and yet checked what version have the problem:

http://bitbucket.org/yuja/redmine-mq-issue4455/src/tip/hg/revisions.diff

Marcel Gmür wrote:

I don't know, what you exactly need, here my full output of the fetch changeset run:

http://pastebin.com/FkVeC5N0

The error only happens, if there is something fetch.

I run now both apps parallel, one is the svn vanilla 0.9-stable, the other the qclone from bitbucket.org, My public address is the uses your patch queue, but I still fetch changesets with the svn version, so I don't get seconds, but it works.

Greets
Ammler

Original Redmine does not have "Repository.find_by_url".
Are you using another patch?

Original Redmine does not have "Repository.find_by_url".

Yes, it has. It's provided by ActiveRecord:

http://www.ruby-forum.com/topic/94068

hg 1.4 fails.

$ hg update 1.4
$ make clean
$ make local

$ hg --version
Mercurial - 分散構成管理ツール(バージョン 1.4)

$ LANG=C ruby script/runner -e test "Repository.find_by_url('/WEB-DOWN/hg-repo/tortoisehg').fetch_changesets" 
/usr/lib/ruby/gems/1.8/gems/rails-2.3.5/lib/rails/gem_dependency.rb:119:Warning: Gem::Dependency#version_requirements is deprecated and will be removed on or after August 2010.  Use #requirement
/usr/lib/ruby/gems/1.8/gems/rails-2.3.5/lib/commands/runner.rb:48: #<REXML::ParseException: No close tag for /log> (REXML::ParseException)
/usr/lib/ruby/1.8/rexml/parsers/treeparser.rb:28:in `parse'
/usr/lib/ruby/1.8/rexml/document.rb:227:in `build'
/usr/lib/ruby/1.8/rexml/document.rb:43:in `initialize'

Toshi Maruyama wrote:

hg 1.4 fails.

Oops, found that "footer" is really new feature:
http://bitbucket.org/mirror/mercurial/changeset/56284451a22c

I'll fix it tomorrow. Thanks for your clarification.

Fixed "missing </log>" error on Mercurial < 1.5.
The latest patches are available at http://bitbucket.org/yuja/redmine-mq-issue4455/src

Thanks Toshi and Marcel.

http://pastebin.com/AKj5tXXX

Mercurial 1.3.1

dev@salieri:~/redmine> hg parent
changeset: 3780:c8f523aaa8fe
branch: 0.9-stable
tag: qtip
tag: hg/version.diff
user: Yuya Nishihara <yuya@tcha.org>
date: Mon Mar 22 16:30:33 2010 +0900
summary: mercurial: get client_version correctly even if LANG is set

hg 1.3 is given by the standard repos of openSUSE 11.2, I try now with 1.5

just like to mention, that after update to 1.5.4, it works how it should.

how would you manage patch queues with redmine?

Patch Branches for Mercurial is interesting.
But pbranch can not import MQ.
And I don't know how to manage target for Redmine svn trunk and 0.9 stable with pbranch.

If we don't use MQ and we use normal hg changesets, I think we can use Transplant extension for Redmine svn trunk and 0.9 stable.

The problem is that redmine does fetch the changesets to the repo I qpushed from the patch queue, what happens, if I pop some out again?

Or if some patches got committed to the branch? Maybe the only solution is not to publish the repo, which the queue is based on?

Marcel Gmür wrote:

Maybe the only solution is not to publish the repo, which the queue is based on?

Indeed. As MQ is designed for private use, you shouldn't publish MQ-applied repository.

I agree with that, but we still need possiblity for qclone, my suggestion is a new SCM type, MQ for example. With that type reposman creates the 2 repos. Repository view automatically shows only the repo in .hg/patches, the other one is ignored from redmine mainly.

I think there is no need to create new SCM type.
One repositry per one project is Redmine limitation #779.
And "stable" and "developement" repository is popular with Mercruial.
Please see #4924 .

Toshi Maruyama wrote:

One repositry per one project is Redmine limitation #779.

Yes, but MQ are 2 repos, which you need only .hg/patches to be public, redmine doesn't need to care about the "base repo", but you need somehow to distinguish for reposman.

Any suggestions, how I can allow my people to create MQ without ssh access?

".hg/patches/.hg" is normal hg repository.
So, you can clone only it.
And "hg qclone" has "-p" option.

$ hg qclone --help

 -p --patches       location of source patch repository

$ hg clone -U http://bitbucket.org/yuja/redmine-mq/.hg/patches yuya-mq
$ ll yuya-mq/.hg
合計 6
-rw-rw-r-- 2 maruyama maruyama   57 2010-06-18 13:16 00changelog.i
-rw-rw-r-- 1 maruyama maruyama   94 2010-06-18 13:17 branchheads.cache
-rw-rw-r-- 1 maruyama maruyama   67 2010-06-18 13:17 hgrc
-rw-rw-r-- 2 maruyama maruyama   23 2010-06-18 13:16 requires
drwxrwxr-x 3 maruyama maruyama 1024 2010-06-18 13:21 store
-rw-rw-r-- 1 maruyama maruyama    7 2010-06-18 13:17 undo.branch
-rw-rw-r-- 1 maruyama maruyama    0 2010-06-18 13:17 undo.dirstate

$ hg qclone -U http://bitbucket.org/svn/redmine -p ssh://localhost/`pwd`/yuya-mq
requesting all changes
adding changesets
adding manifests
adding file changes
added 3757 changesets with 20386 changes to 3133 files (+29 heads)
requesting all changes
adding changesets
adding manifests
adding file changes
added 122 changesets with 292 changes to 73 files

Are you using redmine checkout plugin?
And do you want to show "hg qlone URL" on redmine?
This is redmine checkout plugin issue, not redmine issue.

Toshi Maruyama wrote:

Are you using redmine checkout plugin?
And do you want to show "hg qlone URL" on redmine?
This is redmine checkout plugin issue, not redmine issue.

Sorry, fix typo "qlone" -> "qclone".

Are you using redmine checkout plugin?
And do you want to show "hg qclone URL" on redmine?
This is redmine checkout plugin issue, not redmine issue.

I still don't get, how you will qclone from a redmine project.

Marcel Gmür wrote:

I still don't get, how you will qclone from a redmine project.

Make two projects on redmine¶

One is normal hg repository, another is MQ repositry.

1. tortoisehg-pygtk
2. tortoisehg-pygtk-mq

"hg init" and set repositories at redmine¶

$ hg init /REDMINE/WORK-DIR-NO-RAID/THG-REPOS/tortoisehg-pygtk
$ hg init /REDMINE/WORK-DIR-NO-RAID/THG-REPOS/tortoisehg-pygtk-mq

push to repositories which redmine manages¶

$ mkdir /tmp/redmine-demo
$ hg clone -U http://bitbucket.org/marutosi/tortoisehg-pygtk \
    /tmp/redmine-demo/tortoisehg-pygtk
$ hg clone -U http://bitbucket.org/marutosi/tortoisehg-pygtk-mq/.hg/patches \
    /tmp/redmine-demo/tortoisehg-pygtk-mq

$ hg -R /tmp/redmine-demo/tortoisehg-pygtk push \
    /REDMINE/WORK-DIR-NO-RAID/THG-REPOS/tortoisehg-pygtk
$ hg -R /tmp/redmine-demo/tortoisehg-pygtk-mq push \
    /REDMINE/WORK-DIR-NO-RAID/THG-REPOS/tortoisehg-pygtk-mq

hg qclone¶

$ hg qclone \
      -U /REDMINE/WORK-DIR-NO-RAID/THG-REPOS/tortoisehg-pygtk \
      -p /REDMINE/WORK-DIR-NO-RAID/THG-REPOS/tortoisehg-pygtk-mq

$ hg log -l1
changeset:   6588:6995d295e6f3
tag:         tip
user:        Toshi MARUYAMA <XXXXXXXXXXXXXXXXXXXXX>
date:        Thu Jun 24 15:00:06 2010 +0900
summary:     restore original build env.

$ hg log -l1 --mq
changeset:   1:40bf2cc11197
tag:         tip
user:        Toshi Maruyama <XXXXXXXXXXXXXXXXXXXXX>
date:        Tue Jun 22 19:12:12 2010 +0900
summary:     Manage series.

Is this ready to be reviewed to be included in trunk?

I'm concerned because the latest patches are are targeting 0.9 (should target trunk) and Ammler from IRC pointed me to http://bitbucket.org/yuja/redmine-mq-issue4455/src which looks like a bunch of patches.

BTW, in the current incarnation of the checkout plugin, you can overwrite the command shown in the plugin configuration.

The next version (which is dues soon) will resemble a more github-like UI. So you are then able to define multiple URLs / Protocols.

I would also quite much support the switch to trunk and would of course also continue to test the patch queue on my productive system.

Eric Davis wrote:

I'm concerned because the latest patches are are targeting 0.9 (should target trunk) and Ammler from IRC pointed me to http://bitbucket.org/yuja/redmine-mq-issue4455/src which looks like a bunch of patches.

Yuya's MQ targets both of SVN trunk and 0.9 stable.

Following is SVN trunk applying commands.

$ hg qclone -U http://bitbucket.org/yuja/redmine-mq-issue4455
$ cd redmine-mq-issue4455
$ hg update tip --mq

$ hg qselect -s
+0.9
+experimental
+issue-2664

$ hg qselect
no active guards

$ hg update tip
$ hg branch
default

$ hg pare
changeset:   3809:eb19c68290e6
tag:         tip
user:        edavis10@e93f8b46-1217-0410-a6f0-8f06a7374b81
date:        Wed Jun 30 03:32:18 2010 +0000
summary:     Set @project so macros will work on the welcome and project list. #5781

$ hg qpush -a
applying hg/isodatesec.patch
applying git/changeset-order.diff
applying sort-changesets-by-id.diff
applying changeset-identifier.diff
applying changeset-query-identifier.diff
applying better-format-revision.diff
applying wiki-commit-link-changeset-identifier.diff
applying issue-auto-close-revision-format.diff
applying activity-changeset-identifier.diff
applying repoview-nodeid.diff
applying revision-view-input-field.diff
applying fetch-changesets-catch-command-failed.diff
applying hgext/helper-by-ale.diff
applying hgext/helper-by-me.diff
applying hg/cmd.diff
applying hg/diff.diff
applying hg/annotate.diff
applying hg/revisions.diff
applying hg/summary.diff
applying hg/fetch-changesets.diff
applying hg/latest-changesets.diff
applying hg/tags.diff
applying hg/entries.diff
applying hg/branches.diff
applying hg/version.diff
skipping hg/plain.diff - guarded by ['+experimental']
now at: hg/version.diff

You can export git patch with following command.

$ hg export qbase:qtip --git

I tried to push Yuya's MQ to my github with hg-git, but I could not clone git repository with same following problem.

hg-git issue 117 "Exception on first clone"
http://github.com/schacon/hg-git/issues#issue/117

Eric Davis wrote:

Is this ready to be reviewed to be included in trunk?

Honestly, not yet. Though most parts are okay, some patches contain rough edges like hard-coded SQL, incompatibility with git plugin, etc.

I'm concerned because the latest patches are are targeting 0.9

As Toshi mentioned, it's actually trunk-based. It just includes some changes from trunk to be applied to 0.9 series.

which looks like a bunch of patches.

I'll fold them into two or three when it's ready.

Eric, as mentioned at note-86 and note-87, isodatesec.patch has no risk.
Could you commit this patch?

Maybe you should create own tickets about finished independent patches.

I have also a urgent bug to report:

Projects without any tracker don't work, workaround is to add a tracker, but that isn't needed on vanilla redmine.

Greets
Ammler

snippet from log:

ActionView::TemplateError (Validation failed: Tracker can't be blank, Tracker can't be blank) on line #45 of app/views/layouts/base.rhtml:
42:                                                                                                                                       
43:     <% if display_main_menu?(@project) %>                                                                                             
44:     <div id="main-menu">                                                                                                              
45:         <%= render_main_menu(@project) %>                                                                                             
46:     </div>                                                                                                                            
47:     <% end %>                                                                                                                         
48: </div>                                                                                                                                

    vendor/plugins/redmine_code_review/app/models/code_review_project_setting.rb:37:in `find_or_create'
    vendor/plugins/redmine_code_review/init.rb:52:in `evaluate_init_rb'                                
    lib/redmine/menu_manager.rb:282:in `call'                                                          
    lib/redmine/menu_manager.rb:282:in `allowed_node?'                                                 
    lib/redmine/menu_manager.rb:252:in `menu_items_for'                                                
    lib/redmine/menu_manager.rb:251:in `each'
    lib/redmine/menu_manager.rb:251:in `menu_items_for'
    lib/redmine/menu_manager.rb:176:in `render_menu'
    lib/redmine/menu_manager.rb:166:in `render_main_menu'
    app/views/layouts/base.rhtml:45:in `_run_rhtml_app47views47layouts47base46rhtml'
    passenger (2.2.7) lib/phusion_passenger/rack/request_handler.rb:95:in `process_request'
    passenger (2.2.7) lib/phusion_passenger/abstract_request_handler.rb:207:in `main_loop'
    passenger (2.2.7) lib/phusion_passenger/railz/application_spawner.rb:374:in `start_request_handler'
    passenger (2.2.7) lib/phusion_passenger/railz/application_spawner.rb:332:in `handle_spawn_application'
    passenger (2.2.7) lib/phusion_passenger/utils.rb:184:in `safe_fork'
    passenger (2.2.7) lib/phusion_passenger/railz/application_spawner.rb:330:in `handle_spawn_application'
    passenger (2.2.7) lib/phusion_passenger/abstract_server.rb:352:in `__send__'
    passenger (2.2.7) lib/phusion_passenger/abstract_server.rb:352:in `main_loop'
    passenger (2.2.7) lib/phusion_passenger/abstract_server.rb:196:in `start_synchronously'
    passenger (2.2.7) lib/phusion_passenger/abstract_server.rb:163:in `start'
    passenger (2.2.7) lib/phusion_passenger/railz/application_spawner.rb:209:in `start'
    passenger (2.2.7) lib/phusion_passenger/spawn_manager.rb:262:in `spawn_rails_application'
    passenger (2.2.7) lib/phusion_passenger/abstract_server_collection.rb:126:in `lookup_or_add'
    passenger (2.2.7) lib/phusion_passenger/spawn_manager.rb:256:in `spawn_rails_application'
    passenger (2.2.7) lib/phusion_passenger/abstract_server_collection.rb:80:in `synchronize'
    passenger (2.2.7) lib/phusion_passenger/abstract_server_collection.rb:79:in `synchronize'
    passenger (2.2.7) lib/phusion_passenger/spawn_manager.rb:255:in `spawn_rails_application'
    passenger (2.2.7) lib/phusion_passenger/spawn_manager.rb:154:in `spawn_application'
    passenger (2.2.7) lib/phusion_passenger/spawn_manager.rb:287:in `handle_spawn_application'
    passenger (2.2.7) lib/phusion_passenger/abstract_server.rb:352:in `__send__'
    passenger (2.2.7) lib/phusion_passenger/abstract_server.rb:352:in `main_loop'
    passenger (2.2.7) lib/phusion_passenger/abstract_server.rb:196:in `start_synchronously'

Rendering /home/dev/redmine.hg/public/500.html (500 Internal Server Error)

Marcel Gmür wrote:

Projects without any tracker don't work, workaround is to add a tracker, but that isn't needed on vanilla redmine.

As your log said, it's redmine_code_review's bug, and they've already released the fixed version. See http://www.r-labs.org/issues/392

:-$ shame on me :'-(

Yuya, could you tell us your thought of #5117.

Yuya Nishihara wrote:

incompatibility with git plugin,

Do you mean #5357 ?

Yuya, could you tell us your thought of #5117.

It seems good. IMHO, there's room to clean up more, though.

incompatibility with git plugin,

Do you mean #5357 ?

Yes, git adapter doesn't consider the insertion order.

order by id is fine, just be aware that adding the commits to the db is correct, for example test with commits with exactly same timestamp

Redmine should use the revision number for links per default or else compare the hashes of imported changes with the repo.

As you could rollback, rebase or strip changesets from repos, the hashes and revision numbers can change or the hash could go at all.

Marcel Gmür wrote:

Redmine should use the revision number for links per default or else compare the hashes of imported changes with the repo.

• http://dev.openttdcoop.org/projects/ukrs2-addons/repository/revisions/0
• http://dev.openttdcoop.org/projects/ukrs2-addons/repository/revisions/9ad23a59f7b6

Redmine mercurial mirror on bitbucket stopped updating at r3840. Because it is made by hgsubversion , you can pull from SVN only after r3841 to this hg repository.
I described commands at Open discussion: redmine mercurial mirror on bitbucket .

what we need to do to propose it to 1.0.1?
any issues pending?

Alessio Franceschelli wrote:

what we need to do to propose it to 1.0.1?

It won't go into 1.0.1 but once it's ready it can go into 1.1.0.

Thanks Toshi, I've imported these three tests.
And I decided to enable issue tracker at bitbucket to receive these kind of patches. This thread looks quite long, so I don't want to mess up here just for my patch.

http://bitbucket.org/yuja/redmine-mq-issue4455/issues

Eric Davis wrote:

Alessio Franceschelli wrote:

what we need to do to propose it to 1.0.1?

It won't go into 1.0.1 but once it's ready it can go into 1.1.0.

I guess, chances to get this in trunk might be better to split the queue in some categories and propose those with its own ticket for inclusion. Maybe you can manage that with your subfolders, you already have?

Marcel Gmür wrote:

I guess, chances to get this in trunk might be better to split the queue in some categories and propose those with its own ticket for inclusion. Maybe you can manage that with your subfolders, you already have?

If you can convert the code it into multiple patches I can review and apply individually, then that would be great. For example:

• one patch for branch support
• one patch for tag support
• one patch to fix bug 3421
• ...

If that's possible, some of the bug fixes might be accepted into 1.0.1. The 1.0.x series can't get major features or rewrites since it's "stable" but bug fixes and minor features can still be added.

Yuya Nishihara wrote:

I'll prepare repository on github if it's convenient than patches as attachments.

I pushed my github branch.
http://github.com/marutosi/redmine/commits/hg-patches-svntrunk

• #5117 fix_mercurial_localization_issue.diff
• hg-isodatesec.patch
• hg-changeset-order.patch

Yuya Nishihara wrote:

Eric Davis wrote:

Is this ready to be reviewed to be included in trunk?

Honestly, not yet. Though most parts are okay, some patches contain rough edges like hard-coded SQL, incompatibility with git plugin, etc.

Is this related #6019 and #6159 ?
svn_latest_changesets_improvement.diff at #6159 seems same approach of Yuya's SQL.

Toshi MARUYAMA wrote:

Is this related #6019 and #6159 ?
svn_latest_changesets_improvement.diff at #6159 seems same approach of Yuya's SQL.

Yes. Thanks to it, I was able to rewrite the "hard-coded SQL" patch without sub-query, see #6159.

Each time one is pulling with rebase both the revisions and hashes of local changesets changes and the repository view shows the rebased changests multiple times with new and old revisons and hashes.
Besides the old revisions are now overriding some pulled revisions that never shows in the repository view.
Are these patches also addressing this problem?

Thanks,
Gabriele

Gabriele Garuglieri wrote:

Each time one is pulling with rebase both the revisions and hashes of local changesets changes and the repository view shows the rebased changests multiple times with new and old revisons and hashes.
Besides the old revisions are now overriding some pulled revisions that never shows in the repository view.
Are these patches also addressing this problem?

I described this problem at http://www.redmine.org/issues/3724#note-18 .
I tried to resolve it, but it is very difficult.
Because 'revision' of database is not unique.
I and Yuya agreed that history editing is rare case for public(shared) repository and Redmine does not need treat history editing for public(shared) repository.

I understand that given the constraints with which we have to live, i.e. that the revision and hash, that allow to uniquely identify a changeset in mercurial, can change, the problem of static links in wikis and issues cannot be solved with the current implementation. Perhaps we could think of linking a changeset through a sort of indirection with an internal identifier that could be a hash of committer, date and time of commit, comment and changes that could be used, albeit slowly, to identify and relink again the correct changeset should the history change. But, beside the breaking change in db that this would introduce, this would not solve yet the problem of the static name of the link in pages and i don't know how high could be the probability of hashes collision given the few elements to hash.

Anyhow to begin with i think it could be helpful if it could be integrated in redmine, unless it has some adverse effect that at the moment i don't know, what i do now manually.
After rebase i detect the first out of place revision and then run the following queries:

delete from changes 
where changes.changeset_id in
    (select changesets.id from changesets 
    where changesets.repository_id = <repository_id>
    and changesets.id >= 
        (select changesets.id from changesets
        where changesets.repository_id = <repository_id>
        and revision = 'rev#'
        )
    );

delete from changesets_issues 
where changesets_issues.changeset_id in
    (select changesets.id from changesets 
    where changesets.repository_id = <repository_id>
    and changesets.id >= 
        (select changesets.id from changesets
        where changesets.repository_id = <repository_id>
        and revision = 'rev#'
        )
    );

delete from changesets 
where changesets.repository_id = <repository_id>
and changesets.id >= 
    (select changesets.id from changesets
    where changesets.repository_id = <repository_id>
    and revision = 'rev#'
    );

The next time i visualize the repository it will refetch the missing revisions in the right order. It's a hack but surely faster than redefining the repo and refetching all the revisions.
They will still be visualized in the wrong order since the only sort available is by date but i think this problem is being addressed by one of your patches.

I think that if one knows exactly what he is doing and knows where the history of two repo clones diverged perhaps this could also partially help (static links in pages excluded) for the issue in http://www.redmine.org/issues/3724#note-18 you mentioned first.

Best regards,
Gabriele

Gabriele Garuglieri wrote:

They will still be visualized in the wrong order since the only sort available is by date but i think this problem is being addressed by one of your patches.

Sorry, my patches at #3724 are obsolete.
I am supporting Yuya, now.

Gabriele, please try yuya's MQ.
Yuya's MQ resolves problems that you mention.
http://bitbucket.org/yuja/redmine-mq-issue4455

Toshi MARUYAMA wrote:

Sorry, my patches at #3724 are obsolete.

I deleted my obsolete remote branches of github.
I don't know why does github show http://github.com/marutosi/redmine/commit/30845a8c8ed51a71beb6e1d4df4dfbcf3a55c444 that I described at http://www.redmine.org/issues/3724#note-19 .

As far as I know, redmine 1.0 sets relations between issues and changesets in only case of fetching changesets.
There is a issue "Feature #2009 Manually add related revisions".

If '#issue_number" is written in commit log, redmine add 'rNN' at a journal.
I think it is a problem for Mercurial.
Yuya's MQ changes from 'rNN' to 'commit:AAA' for it at http://bitbucket.org/yuja/redmine-mq-issue4455/src/beeb014b9a89/hg-use-scmid.patch/issue-auto-close.diff.

bitbucket.org/svn/redmine seems stalled

I had running my own hg convert already at http://hg.openttdcoop.org/redmine and could push it to bitbucket.org/OpenTTD/redmine, maybe you like to use that, we could also use our redmine, which is based on the MQ for the project management, if you like.

http://dev.openttdcoop.org/projects/redmine
we have already HGRedmine running for push authentification

Please tell me, if there is interest...

Thanks, Marcel.

I don't like "hg convert", because we need to keep .hg/shamap as described at http://www.redmine.org/boards/1/topics/12312#message-15863 .
And I don't need old branches.

I start cron job of redmine svn mirror and pushed following repositories.

• http://bitbucket.org/marutosi/redmine-hgsubversion-allbranches-clean
• http://bitbucket.org/marutosi/redmine-hgsubversion-svntrunk-clean
• http://bitbucket.org/marutosi/redmine-hgsubversion-1.0-clean

But my cron job works only when my Fedora is living.

something else:

tip is not always head of default, it can also be head of a branch.

It should somehow be viewable, which branch we currently see, maybe by showing always a branch in the branch menu item, as there isn't a empty branch anyway.

http://dev.openttdcoop.org/projects/redmine
we have already HGRedmine running for push authentification

Please tell me, if there is interest...

Absolutely.

Alessio Caiazza wrote:

Today I made a patch, based on this branch, for editing some options of .hg/hgrc inside project repository settings.

This patch needs some more work, but I'll publish it very soon (I hope).

... 7 months later ...

I'm too fast ;)
As I promised, Feature #6515 (hgrc editor)

Marcel Gmür wrote:

Maybe you should create own tickets about finished independent patches.

I created new issue for hg-isodatesec.patch. New issue is #6656 .

Yuya Nishihara wrote:

Currently these two patches are ready. Could you review them?

• hg-isodatesec.patch ... more accurate commit timestamp, by Toshi
• hg-changeset-order.patch ... sort changesets by revision, fixes #3449, #3567
  (NOTE: It contains git binary diff)

And I think fix_mercurial_localization_issue.diff of #5117 is good candidate.

I'll prepare repository on github if it's convenient than patches as attachments.

I finished updating these issues status.
All these patches include unit tests and independent for each other.
Please set target of these issues to next stable 1.0.3 or 1.1.

1. Mercurial adapter loses seconds of commit times¶

• #6656

• http://bitbucket.org/yuja/redmine-mq-issue4455/src/d6bc15c54f5d/hg-isodatesec.patch

• http://github.com/marutosi/redmine/commits/hg-isodatesec
• http://github.com/marutosi/redmine/commits/c60463816bd02fbadb61
• http://github.com/marutosi/redmine/commit/c60463816bd02fbadb61

• http://github.com/edavis10/redmine/pull/9

2. Sort changesets by revision and fix fetching performance problem¶

• #3449, #3567

• http://bitbucket.org/yuja/redmine-mq-issue4455/src/d6bc15c54f5d/hg-changeset-order.patch

Github revision

Because I pushed by hg-git, author and commit date are 2010-07-29.
So, github commit ordering is strange.
This is same problem with #5357.

• http://github.com/marutosi/redmine/commits/hg-order-yuya-mq-c5a47a5f63
• http://github.com/marutosi/redmine/commits/9ea7a1d1fc317f79c8f3730b11de2f2b8fd93f43
• http://github.com/marutosi/redmine/commit/9ea7a1d1fc317f79c8f3730b11de2f2b8fd93f43

• http://github.com/edavis10/redmine/pull/11

3. Mercurial_adapter should ensure the right LANG environment variable¶

Issue

• #5117

• http://bitbucket.org/yuja/redmine-mq-issue4455/src/d6bc15c54f5d/hg/version.diff

• http://github.com/marutosi/redmine/commits/hg-version-yuya-mq-c5a47a5f63
• http://github.com/marutosi/redmine/commits/e2334dcb4b85976a1224
• http://github.com/marutosi/redmine/commit/e2334dcb4b85976a1224

• http://github.com/edavis10/redmine/pull/8

Eric Davis wrote:

You think it's ready to be included in Redmine 1.1?

Yes.

"1. Mercurial adapter loses seconds of commit times (#6656)" and "3. Mercurial_adapter should ensure the right LANG environment variable(#5117)" are bug fix with no behaviour change, so I wish to set target next stable 1.0.3.

"2. Sort changesets by revision and fix fetching performance problem (#3449, #3567)" is bug fix with behaviour change.
But this patch effect only for Mercrurial and does not effect other SCM.
I think all Mercurial user can accept this behaviour change.
Please judge set target 1.0.3 or 1.1.

This overhaul(#4455) has many issues.
http://bitbucket.org/yuja/redmine-mq-issue4455/issues?status=new&status=open
We still have many works.

Toshi MARUYAMA wrote:

As far as I know, redmine 1.0 sets relations between issues and changesets in only case of fetching changesets.
There is a issue "Feature #2009 Manually add related revisions".

If '#issue_number" is written in commit log, redmine add 'rNN' at a journal.
I think it is a problem for Mercurial.
Yuya's MQ changes from 'rNN' to 'commit:AAA' for it at http://bitbucket.org/yuja/redmine-mq-issue4455/src/beeb014b9a89/hg-use-scmid.patch/issue-auto-close.diff.

I created new issue for this problem. New issue is #6681.

I believe issue #6892 should be taken into consideration as part of the overhaul.

Sam Kuper wrote:

I believe issue #6892 should be taken into consideration as part of the overhaul.

reposman.rb supports to create Mercurial repository.
Please see note-88 of this issue.

shouldn't you apply this mq to the repo http://bitbucket.org/marutosi/redmine-hgsubversion-allbranches-clean since the current one is "dead"?

Marcel Gmür wrote:

shouldn't you apply this mq to the repo http://bitbucket.org/marutosi/redmine-hgsubversion-allbranches-clean since the current one is "dead"?

My hourly cron job is running only while my machine is alive.
Now I have executed this job by hand.

Marcel, are you synchronizing my repository and your repository?
Following commit id are same.

• http://bitbucket.org/marutosi/redmine-hgsubversion-allbranches-clean/changeset/5d4f1297553f
• http://svn.dev.openttdcoop.org/projects/redmine/repository/revisions/4435

The following is main part of my hourly cron job.

hg -q -R ${ALLDIR} pull http://bitbucket.org/${BBUSER}/redmine-hgsubversion-allbranches-clean 
hg -q -R ${ALLDIR} svn rebuildmeta http://redmine.rubyforge.org/svn/
hg -q -R ${ALLDIR} pull http://redmine.rubyforge.org/svn/
hg -q -R ${ALLDIR} push https://${BBUSER}:${BBPW}@bitbucket.org/${BBUSER}/redmine-hgsubversion-allbranches-clean
hg -q -R ${ALLDIR} push -b default https://${BBUSER}:${BBPW}@bitbucket.org/${BBUSER}/redmine-hgsubversion-svntrunk-clean
hg -q -R ${ALLDIR} push -b 1.0-stable https://${BBUSER}:${BBPW}@bitbucket.org/${BBUSER}/redmine-hgsubversion-1.0-clean

Yes, I changed my paths now and sync your repo and the MQ only (no qclone anymore)

dev.openttdcoop.org runs on a quite stable public server, as it is the official place for many addons of OpenTTD, if you like we could run a convert there and push to bitbucket.

I don't care to use hg convert or hgsubversion, both need a support file, btw.

Edit:

http://svn.dev.openttdcoop.org is the clean svn (trunk) version of Redmine
http://dev.openttdcoop.org is mainly this MQ and some minor patches
(both use the same DB)

Marcel Gmür wrote:

dev.openttdcoop.org runs on a quite stable public server, as it is the official place for many addons of OpenTTD, if you like we could run a convert there and push to bitbucket.

I wish so.

I setup now a new hgsubversion convert to https://bitbucket.org/redmine/redmine, also the hgsubversion metdadate are uploaded, so it would be quite easy for someone else to continue the convert, whenever something on my side fails.

Also the user redmine is made just for this, if someone of the officials feels, I am happy to handover that account.

the sync runs in hourly base, I don't think, we need the branches alone, as you can use "hg pull -b <branch>", but if you like clean branches, tell me.

I have also included the redmine convert to our Redmine which does use this MQ, if you wish, you are welcome to develop there, I can give you access to our redmine which does give you the ability to create own projects and repos...

Something else? Please tell or mail to redmine@ammler.ch

stupid me, didn't see that you can rebuild the metadata..., fixed and also a trunk and 1.0 repo will be synced...

do you now create the mq from there?

Revision 0 of original hgsubversion mirror and my repository are 711945aa86b2 .
Revision 0 of your repository is 108d86396bc7 .
Did you create new revision from revesion 0?

Are you using author map?
Commit user of original hgsubversion mirror is "jplang@e93f8b46-1217-0410-a6f0-8f06a7374b81".
And are you using another setting? Can you publish it?

Python PEP 385 "Migrating from svn to Mercurial" publishes scripts of hgsubversion at http://hg.python.org/pymigr .
And this scripts use author-map .

I am using no setting in current hgsubversion pulling.
So, commit user is "jplang@e93f8b46-1217-0410-a6f0-8f06a7374b81".
Redmine Subversion branch layout is standard which is trunk, branches and tags.

I think this issue should be a feature tracker, rather than a patch tracker.

Toshi: hmm, true, didn't think it is necessary to have same hashes, so I removed the (imo) unnecessary host uid, my settings are:
https://bitbucket.org/redmine/redmine/wiki (no authormap, just removing the host)

bo ye: this issue is not just one patch, it is a quite big Patch Queue ;-)

(the uuid can still be found in .hg/svn/uuid)

I found new original Redmine problem and create new issue #7064.

Marcel Gmür wrote:

bo ye: this issue is not just one patch, it is a quite big Patch Queue ;-)

I just wish people could see this issue on the roadmap page of redmine(no need to click into unplanned page), so there would be more up up votes. :)

The problem is simply from what I get in the IRC channel, no active contributor uses mecurial, so this really needs to work quite well for any progress. But Toshi and Yuya do a very good job on maintaining the MQ. I run the MQ on our productive system now for quite some time already without any issues. (Thanks to no db changes, mostly.)

Marcel Gmür wrote:

(Thanks to no db changes, mostly.)

Patch at note-2 of #7064 has db changes.

Mercurial test repository at note-2 of #7064 contains two branches and two tags.
And it has 'Ü' in latin-1 not UTF-8.
And it has '%' and '_' for SQL 'like' escape test.

I wish to be committed the patch at note-2 of #7064 to Redmine SVN trunk firstly.
Please review it.

Hi, thanks to the effort of Toshi and Marcel, we provide the snapshot release of Redmine with whole patches applied:

hg clone https://bitbucket.org/redmine/redmine-issue4455-snapshot/#1.0-issue4455

I'll keep it up-to-date until these patches can be merged into Redmine 1.1 or 1.2?

r4539:r4542 will break some patches in the queue (hg/cmd & hg/diff at least)

Marcel Gmür wrote:

r4539:r4542 will break some patches in the queue (hg/cmd & hg/diff at least)

Rebased onto r4542. Please try the latest MQ patches.

I saw plans that Redmine 1.1 will be released in the next few weeks at Plans for next Redmine releases .

I wish to set target of following two patches to Redmine 1.1.
These patches have unit tests and functional tests.

1. Sort changesets by revision, fetching performance problem and file name encoding¶

Patch

• Note-2 of #7064
  • hg-2010-12-10.diff
  • mercurial_repository.tar.gz

Issue

• #2664, #5408, #3449, #3567, #7064

2. Changing revision label and identifier at SCM adapter level¶

Patch

• Note-23 and Note-25 of #6092
  • revision-truncate-20101213.diff
  • revision-truncate-20101213-fix-fixtures.diff

Issue

• #3244, #3945, #6092

Toshi MARUYAMA wrote:

I wish to set target of following two patches to Redmine 1.1.
These patches have unit tests and functional tests.

1. Sort changesets by revision, fetching performance problem and file name encoding¶

Patch

• Note-2 of #7064
  • hg-2010-12-10.diff
  • mercurial_repository.tar.gz

Hi, they're different issue.
"sort changesets by revision" is simple enough for 1.1, but IMO "file name encoding" isn't.

I can see watchers list of issues.
I notice this issue watchers list is empty.
Why?

Toshi Maruyama wrote:

I can see watchers list of issues.
I notice this issue watchers list is empty.
Why?

I see 3 observers in the sidebar for this ticket.

Jean-Philippe Lang wrote:

Toshi Maruyama wrote:

I can see watchers list of issues.
I notice this issue watchers list is empty.
Why?

I see 3 observers in the sidebar for this ticket.

Yes.
At I wrote note 194, watchers list was empty.
So, I wrote a message at http://www.redmine.org/boards/1/topics/12312?r=20538#message-20538 .

He :-)

I don't get, why empty watchers list is an error, personally I read the rss feed from this ticket...

I finished committing to Redmine 1.1 about Mercurial.
For a while, I focus other SCMs (Darcs, CVS and Bazaar).
Yuya, please update your MQ.

Yes, I think we're done with 1.1 (release planned this week-end).
Can we close this ticket and split what is left in specific tickets? This thread becomes hard to read.

Jean-Philippe Lang wrote:

Can we close this ticket and split what is left in specific tickets? This thread becomes hard to read.

I think so, too. I will summarize Mercurial issues.

Toshi MARUYAMA wrote:

Yuya, please update your MQ.

Thanks. Rebased MQ to SVN trunk r4651 and dropped empty/needless patches.
Note that MQ patches no longer support 1.0-stable. And it can be broken due to possible bad conflict resolution.

https://bitbucket.org/yuja/redmine-mq-issue4455/

Jean-Philippe Lang wrote:

Can we close this ticket and split what is left in specific tickets? This thread becomes hard to read.

+1

Mercurial overhaul next steps¶

1. Defect #3724 change revision label and identifier
2. Defect #3421 browse directory tree
3. Feature #1981 tag
4. Feature #7246 "named branch"

Please add notes.

A little proposal for the Mercurial overhaul MQ: instead using guard for the branch 1.1-stable, I would use also branch in the MQ, it does make it easier for keeping them working for branches.

I will close #3724 and focus #3421.
I have some questions about redminehelper.py .

• Is copyright correct?
  Please see source:tags/1.1.0/app/models/repository/git.rb
• How to deal redminehelper.pyo and redminehelper.pyc?
  Debian distributes Redmine package.
  http://packages.debian.org/sid/redmine
  What happens if redmine process does not have permission to write directory?

Yuya's MQ contains fixing #3421 and latest changesets improvement.
Could you change order in series?

Marcel Gmür wrote:

A little proposal for the Mercurial overhaul MQ: instead using guard for the branch 1.1-stable, I would use also branch in the MQ, it does make it easier for keeping them working for branches.

Hmm, it could be, but I'm too lazy to manage multiple branches.

Toshi MARUYAMA wrote:

I have some questions about redminehelper.py .

• Is copyright correct?

I think it's correct. If your concern is wording, it can be changed to follow the Redmine's coding style.

How to deal redminehelper.pyo and redminehelper.pyc?
What happens if redmine process does not have permission to write directory?

Without write permission, python silently generates nothing.

Could you change order in series?

Yeah, I should do it. But, as I mentioned before, I want to rewrite redminehelper.py to use XML instead of original format.

Yuya Nishihara wrote:

Marcel Gmür wrote:

A little proposal for the Mercurial overhaul MQ: instead using guard for the branch 1.1-stable, I would use also branch in the MQ, it does make it easier for keeping them working for branches.

Hmm, it could be, but I'm too lazy to manage multiple branches.

you really think, it is easier to mess with guards?

Somthing elese related to the the branches and tags menu: Sorting should be DESC, newest branch/tag on top, I guess like bitbucket...

If fixing #3421 requires redminehelper.py, "extra/mercurial/redminehelper.py" path is strange.
Should we relocate path as following?

lib/redmine/scm/adapters
mercurial
├── ext
│   └── redminehelper.py
└── xml
    ├── hg-template-0.9.5.tmpl
    └── hg-template-1.0.tmpl

Marcel Gmür wrote:

Somthing elese related to the the branches and tags menu: Sorting should be DESC, newest branch/tag on top, I guess like bitbucket...

Oops, it looks to be a random order because of Hash. I'll fix it later, thanks!

Toshi MARUYAMA wrote:

lib/redmine/scm/adapters

mercurial
├── ext
│   └── redminehelper.py
└── xml
    ├── hg-template-0.9.5.tmpl
    └── hg-template-1.0.tmpl

Not sure, but isn't it enough like this?

mercurial/
+ redminehelper.py
+ hg-template-0.9.5.tmpl
+ hg-template-1.0.tmpl

Marcel Gmür wrote:

A little proposal for the Mercurial overhaul MQ: instead using guard for the branch 1.1-stable, I would use also branch in the MQ, it does make it easier for keeping them working for branches.

I forked MQ patches for 1.1-stable because it becomes headache to keep them up-to-date. :)
From now, patches for 1.1-stable are maintained separately, no improvements will go for them.

I create new issue #7597.

I want to rewrite redminehelper.py to use XML instead of original format.

Now redminehelper.py generates XML and updated mercurial_adapter.rb to use ActiveSupport::XmlMini:
https://bitbucket.org/yuja/redmine-mq-issue4455/

Should I move redminehelper.py under lib/redmine/scm/adapters/mercurial?

Yuya Nishihara wrote:

Should I move redminehelper.py under lib/redmine/scm/adapters/mercurial?

I think note-208 is much better.

Toshi MARUYAMA wrote:

I think note-208 is much better.

of which?

lib/redmine/scm/adapters/mercurial/ext/redminehelper.py
or
lib/redmine/scm/adapters/mercurial/redminehelper.py ?

Yuya Nishihara wrote:

of which?

lib/redmine/scm/adapters/mercurial/ext/redminehelper.py
or
lib/redmine/scm/adapters/mercurial/redminehelper.py ?

lib/redmine/scm/adapters/mercurial/redminehelper.py

I create new issue #7618.

Toshi MARUYAMA wrote:

lib/redmine/scm/adapters/mercurial/redminehelper.py

Thanks for clarify. Moved it under adapters/mercurial directory.

I committed 'hg' method in r4830.
Does "--cwd" work fine in Windows UNC path such as "\\server\folder"?

source:trunk/lib/redmine/scm/adapters/mercurial_adapter.rb@4830#L234

Toshi MARUYAMA wrote:

Does "--cwd" work fine in Windows UNC path such as "\\server\folder"?

It works for me, on Mercurial 1.7.2. What error you've got on log?

TortoiseHg shellext checks UNC or not.
https://bitbucket.org/tortoisehg/stable/src/9be162b05968/win32/shellext/TortoiseUtils.cpp#cl-184

Redmine 1.1 uses only one "--cwd".
source:tags/1.1.1/lib/redmine/scm/adapters/mercurial_adapter.rb#L83

In general, changing directory is very dangerous.

why is --cwd needed at all since you don't work with working copies at all

Toshi MARUYAMA wrote:

TortoiseHg shellext checks UNC or not.

Hmm, but I don't think -R accepts UNC but --cwd doesn't. Isn't it strange?

In general, changing directory is very dangerous.

by its context. It's pretty safe to change the working directory of child 'hg' process.
There's no threading issue.

Marcel Gmür wrote:

why is --cwd needed at all

To avoid concatenation of repo filesystem path and inner-repo path:

$ hg -R /path/to/repo log /path/to/repo/foo
...
$ hg --cwd /path/to/repo log foo
...

I changed "--cwd" to "-R", and all adapter lib tests passes.
The remaining work is that latest_changesets supports tags and branches.

I start to hack #2664.
Non UTF-8 sub directory (EUC-JP) can not show latest_changesets.
I don't know the reason.

As I wrote in r5091 comment, MySQL test on CI server fails.

http://www.redmine.org/builds/index.html

  1) Failure:
test_latest_changesets(RepositoryMercurialTest) [/test/unit/repository_mercurial_test.rb:90]:
<["28", "27"]> expected but was
<["0", "1"]>.

  2) Failure:
test_latest_changesets_with_limit(RepositoryMercurialTest) [/test/unit/repository_mercurial_test.rb:198]:
<[#<Changeset id: 797, repository_id: 64, revision: "28", committer: "test Ü <test00@foo.bar>", committed_on: "2007-10-31 17:00:00", comments: "commit default branch.", commit_date: "2007-10-31", scmid: "3ae45e2d177d", user_id: nil>,
 #<Changeset id: 796, repository_id: 64, revision: "27", committer: "jsmith <jsmith@foo.bar>", committed_on: "1980-12-31 17:00:00", comments: "copy latin-1 path files to latin-1 subdir.", commit_date: "1980-12-31", scmid: "7bbf4c738e71", user_id: 2>]> expected but was
<[#<Changeset id: 769, repository_id: 64, revision: "0", committer: "jsmith <jsmith@foo.bar>", committed_on: "2007-12-14 10:22:52", comments: "Initial import.\nThe repository contains 3 files.", commit_date: "2007-12-14", scmid: "0885933ad4f6", user_id: 2>,
 #<Changeset id: 770, repository_id: 64, revision: "1", committer: "jsmith <jsmith@foo.bar>", committed_on: "2007-12-14 10:24:01", comments: "Added 2 files and modified one.", commit_date: "2007-12-14", scmid: "9d5b5b004199", user_id: 2>]>.