phases: rework the logic of _pushdiscoveryphase to bound complexity
This rework the various graph traversal in _pushdiscoveryphase to keep the
complexity in check.
This is done though a couple of things:
- first, limiting the space we have to explore, for example, if we are not in
publishing push, we don't need to consider remote draft roots that are also
draft locally, as there is nothing to be moved there.
- avoid unbounded descendant computation, and use the faster "rev between"
computation.
This provide a massive boost to performance when exchanging with repository with
a massive amount of draft, like mozilla-try:
### data-env-vars.name = mozilla-try-2023-03-22-zstd-sparse-revlog
# benchmark.name = hg.command.push
# bin-env-vars.hg.flavor = default
# bin-env-vars.hg.py-re2-module = default
# benchmark.variants.explicit-rev = all-out-heads
# benchmark.variants.issue6528 = disabled
# benchmark.variants.protocol = ssh
# benchmark.variants.reuse-external-delta-parent = default
## benchmark.variants.revs = any-1-extra-rev
before: 20.346590 seconds
after: 11.232059 seconds (-38.15%, -7.48 seconds)
## benchmark.variants.revs = any-100-extra-rev
before: 24.752051 seconds
after: 15.367412 seconds (-37.91%, -9.38 seconds)
After this changes, the push operation is still quite too slow. Some of this
can be attributed to general phases slowness (reading all the roots from disk
for example) and other know slowness (not using persistent-nodemap, branchmap,
tags, etc. We are also working on them, but with this series, phase discovery
during push no longer showing up in profile and this is a pretty nice and bit
low-hanging fruit out of the way.
### (same case as the above)
# benchmark.variants.revs = any-1-extra-rev
pre-%ln-change: 44.235070
this-changeset: 11.232059 seconds (-74.61%, -33.00 seconds)
# benchmark.variants.revs = any-100-extra-rev
pre-%ln-change: 49.234697
this-changeset: 15.367412 seconds (-68.79%, -33.87 seconds)
Note that with this change, the `hg push` performance is now much closer to the
`hg pull` performance, even it still lagging behind a bit. (and the overall
performance are still too slow).
### data-env-vars.name = mozilla-try-2023-03-22-ds2-pnm
# benchmark.variants.explicit-rev = all-out-heads
# benchmark.variants.issue6528 = disabled
# benchmark.variants.protocol = ssh
# benchmark.variants.pulled-delta-reuse-policy = default
# bin-env-vars.hg.flavor = rust
## benchmark.variants.revs = any-1-extra-rev
hg.command.pull: 6.517450
hg.command.push: 11.219888
## benchmark.variants.revs = any-100-extra-rev
hg.command.pull: 10.160991
hg.command.push: 14.251107
### data-env-vars.name = mozilla-try-2023-03-22-zstd-sparse-revlog
# bin-env-vars.hg.py-re2-module = default
# benchmark.variants.explicit-rev = all-out-heads
# benchmark.variants.issue6528 = disabled
# benchmark.variants.protocol = ssh
# benchmark.variants.pulled-delta-reuse-policy = default
## bin-env-vars.hg.flavor = default
## benchmark.variants.revs = any-1-extra-rev
hg.command.pull: 8.577772
hg.command.push: 11.232059
## bin-env-vars.hg.flavor = default
## benchmark.variants.revs = any-100-extra-rev
hg.command.pull: 13.152976
hg.command.push: 15.367412
## bin-env-vars.hg.flavor = rust
## benchmark.variants.revs = any-1-extra-rev
hg.command.pull: 8.731982
hg.command.push: 11.178751
## bin-env-vars.hg.flavor = rust
## benchmark.variants.revs = any-100-extra-rev
hg.command.pull: 13.184236
hg.command.push: 15.620843
Tests for access level on hidden commits by various commands on based of their
type.
Setting the required config to start this
$ cat >> $HGRCPATH <<EOF
> [experimental]
> evolution=createmarkers, allowunstable
> directaccess=True
> directaccess.revnums=True
> [extensions]
> amend =
> EOF
$ hg init repo
$ cd repo
$ for ch in a b c; do touch $ch; echo "foo" >> $ch; hg ci -Aqm "Added "$ch; done
$ hg log -G -T '{rev}:{node} {desc}' --hidden
@ 2:28ad74487de9599d00d81085be739c61fc340652 Added c
|
o 1:29becc82797a4bc11ec8880b58eaecd2ab3e7760 Added b
|
o 0:18d04c59bb5d2d4090ad9a5b59bd6274adb63add Added a
$ echo "bar" >> c
$ hg amend
$ hg log -G -T '{rev}:{node} {desc}' --hidden
@ 3:2443a0e664694756d8b435d06b6ad84f941b6fc0 Added c
|
| x 2:28ad74487de9599d00d81085be739c61fc340652 Added c
|/
o 1:29becc82797a4bc11ec8880b58eaecd2ab3e7760 Added b
|
o 0:18d04c59bb5d2d4090ad9a5b59bd6274adb63add Added a
Testing read only commands on the hidden revision
Testing with rev number
$ hg exp 2 --config experimental.directaccess.revnums=False
abort: hidden revision '2' was rewritten as: 2443a0e66469
(use --hidden to access hidden revisions)
[10]
$ hg exp 2
# HG changeset patch
# User test
# Date 0 0
# Thu Jan 01 00:00:00 1970 +0000
# Node ID 28ad74487de9599d00d81085be739c61fc340652
# Parent 29becc82797a4bc11ec8880b58eaecd2ab3e7760
Added c
diff -r 29becc82797a -r 28ad74487de9 c
--- /dev/null Thu Jan 01 00:00:00 1970 +0000
+++ b/c Thu Jan 01 00:00:00 1970 +0000
@@ -0,0 +1,1 @@
+foo
$ hg log -r 2
changeset: 2:28ad74487de9
user: test
date: Thu Jan 01 00:00:00 1970 +0000
obsolete: rewritten using amend as 3:2443a0e66469
summary: Added c
$ hg identify -r 2
28ad74487de9
$ hg status --change 2
A c
$ hg status --change 2 --config experimental.directaccess.revnums=False
abort: hidden revision '2' was rewritten as: 2443a0e66469
(use --hidden to access hidden revisions)
[10]
$ hg diff -c 2
diff -r 29becc82797a -r 28ad74487de9 c
--- /dev/null Thu Jan 01 00:00:00 1970 +0000
+++ b/c Thu Jan 01 00:00:00 1970 +0000
@@ -0,0 +1,1 @@
+foo
Testing with hash
`hg export`
$ hg exp 28ad74
# HG changeset patch
# User test
# Date 0 0
# Thu Jan 01 00:00:00 1970 +0000
# Node ID 28ad74487de9599d00d81085be739c61fc340652
# Parent 29becc82797a4bc11ec8880b58eaecd2ab3e7760
Added c
diff -r 29becc82797a -r 28ad74487de9 c
--- /dev/null Thu Jan 01 00:00:00 1970 +0000
+++ b/c Thu Jan 01 00:00:00 1970 +0000
@@ -0,0 +1,1 @@
+foo
`hg log`
$ hg log -r 28ad74
changeset: 2:28ad74487de9
user: test
date: Thu Jan 01 00:00:00 1970 +0000
obsolete: rewritten using amend as 3:2443a0e66469
summary: Added c
`hg cat`
$ hg cat -r 28ad74 c
foo
`hg diff`
$ hg diff -c 28ad74
diff -r 29becc82797a -r 28ad74487de9 c
--- /dev/null Thu Jan 01 00:00:00 1970 +0000
+++ b/c Thu Jan 01 00:00:00 1970 +0000
@@ -0,0 +1,1 @@
+foo
`hg files`
$ hg files -r 28ad74
a
b
c
`hg identify`
$ hg identify -r 28ad74
28ad74487de9
`hg annotate`
$ hg annotate -r 28ad74 a
0: foo
`hg status`
$ hg status --change 28ad74
A c
`hg archive`
This should not throw error
$ hg archive -r 28ad74 foo
`hg update`
$ hg up 28ad74
1 files updated, 0 files merged, 0 files removed, 0 files unresolved
updated to hidden changeset 28ad74487de9
(hidden revision '28ad74487de9' was rewritten as: 2443a0e66469)
$ hg up 3
1 files updated, 0 files merged, 0 files removed, 0 files unresolved
$ hg up
0 files updated, 0 files merged, 0 files removed, 0 files unresolved
`hg revert`
$ hg revert -r 28ad74 --all
reverting c
$ hg diff
diff -r 2443a0e66469 c
--- a/c Thu Jan 01 00:00:00 1970 +0000
+++ b/c Thu Jan 01 00:00:00 1970 +0000
@@ -1,2 +1,1 @@
foo
-bar
Test special hash/rev
$ hg log -qr 'null:wdir() & 000000000000'
-1:000000000000
$ hg log -qr 'null:wdir() & ffffffffffff'
2147483647:ffffffffffff
$ hg log -qr 'null:wdir() & rev(-1)'
-1:000000000000
$ hg log -qr 'null:wdir() & rev(2147483647)'
2147483647:ffffffffffff
$ hg log -qr 'null:wdir() & 2147483647'
2147483647:ffffffffffff
Commands with undefined intent should not work right now
$ hg phase -r 28ad74
abort: hidden revision '28ad74' was rewritten as: 2443a0e66469
(use --hidden to access hidden revisions)
[10]
$ hg phase -r 2
abort: hidden revision '2' was rewritten as: 2443a0e66469
(use --hidden to access hidden revisions)
[10]
Setting a bookmark will make that changeset unhidden, so this should come in end
$ hg bookmarks -r 28ad74 book
bookmarking hidden changeset 28ad74487de9
(hidden revision '28ad74487de9' was rewritten as: 2443a0e66469)
$ hg bookmarks
book 2:28ad74487de9