revset: stop serializing node when using "%ln"
Turning hundred of thousand of node from node to hex and back can be slow… what
about we stop doing it?
In many case were we are using node id we should be using revision id. However
this is not a good reason to have a stupidly slow implementation of "%ln".
This caught my attention again because the phase discovery during push make an
extensive use of "%ln" or huge set. In absolute, that phase discovery probably
should use "%ld" and need to improves its algorithmic complexity, but improving
"%ln" seems simple and long overdue. This greatly speeds up `hg push` on
repository with many drafts.
Here are some relevant poulpe benchmarks:
### data-env-vars.name = mozilla-try-2023-03-22-zstd-sparse-revlog
# benchmark.name = hg.command.push
# bin-env-vars.hg.flavor = default
# bin-env-vars.hg.py-re2-module = default
# benchmark.variants.explicit-rev = all-out-heads
# benchmark.variants.issue6528 = disabled
# benchmark.variants.protocol = ssh
# benchmark.variants.reuse-external-delta-parent = default
## benchmark.variants.revs = any-1-extra-rev
before: 44.235070
after: 20.416329 (-53.85%, -23.82)
## benchmark.variants.revs = any-100-extra-rev
before: 49.234697
after: 26.519829 (-46.14%, -22.71)
### benchmark.name = hg.command.bundle
# bin-env-vars.hg.flavor = default
# bin-env-vars.hg.py-re2-module = default
# benchmark.variants.revs = all
# benchmark.variants.type = none-streamv2
## data-env-vars.name = heptapod-public-2024-03-25-zstd-sparse-revlog
before: 10.138396
after: 7.750458 (-23.55%, -2.39)
## data-env-vars.name = mercurial-public-2024-03-22-zstd-sparse-revlog
before: 1.263859
after: 0.700229 (-44.60%, -0.56)
## data-env-vars.name = mozilla-try-2023-03-22-zstd-sparse-revlog
before: 399.484481
after: 346.5089 (-13.26%, -52.98)
## data-env-vars.name = pypy-2024-03-22-zstd-sparse-revlog
before: 4.540080
after: 3.401700 (-25.07%, -1.14)
## data-env-vars.name = tryton-public-2024-03-22-zstd-sparse-revlog
before: 2.975765
after: 1.870798 (-37.13%, -1.10)
=====================================================
test behavior of the `internal` phase around bundling
=====================================================
Long story short, internal changeset are internal implementation details and
they should never leave the repository. Hence, they should never be in a
bundle.
Setup
=====
$ cat << EOF >> $HGRCPATH
> [ui]
> logtemplate="{node|short} [{phase}] {desc|firstline}"
> EOF
$ hg init reference-repo --config format.use-internal-phase=yes
$ cd reference-repo
$ echo a > a
$ hg add a
$ hg commit -m "a"
$ echo b > b
$ hg add b
$ hg commit -m "b"
$ echo b > c
$ hg add c
$ hg commit -m "c"
$ hg log -G
@ 07f0cc02c068 [draft] c
|
o d2ae7f538514 [draft] b
|
o cb9a9f314b8b [draft] a
$ hg up ".^"
0 files updated, 0 files merged, 1 files removed, 0 files unresolved
do a shelve
$ touch a_file.txt
$ hg shelve -A
adding a_file.txt
shelved as default
0 files updated, 0 files merged, 1 files removed, 0 files unresolved
$ hg log -G --hidden
o 2ec3cf310d86 [internal] changes to: b
|
| o 07f0cc02c068 [draft] c
|/
@ d2ae7f538514 [draft] b
|
o cb9a9f314b8b [draft] a
$ shelved_node=`hg log --rev tip --hidden -T '{node|short}'`
add more changeset above it
$ hg up 'desc(a)'
0 files updated, 0 files merged, 1 files removed, 0 files unresolved
$ echo d > d
$ hg add d
$ hg commit -m "d"
created new head
$ echo d > e
$ hg add e
$ hg commit -m "e"
$ hg up null
0 files updated, 0 files merged, 3 files removed, 0 files unresolved
$ hg log -G
o 636bc07920e3 [draft] e
|
o 980f7dc84c29 [draft] d
|
| o 07f0cc02c068 [draft] c
| |
| o d2ae7f538514 [draft] b
|/
o cb9a9f314b8b [draft] a
$ hg log -G --hidden
o 636bc07920e3 [draft] e
|
o 980f7dc84c29 [draft] d
|
| o 2ec3cf310d86 [internal] changes to: b
| |
| | o 07f0cc02c068 [draft] c
| |/
| o d2ae7f538514 [draft] b
|/
o cb9a9f314b8b [draft] a
$ cd ..
backup bundle from strip
========================
strip an ancestors of the internal changeset
--------------------------------------------
$ cp -aR reference-repo strip-ancestor
$ cd strip-ancestor
The internal change is stripped, yet it should be skipped from the backup bundle.
$ hg log -G
o 636bc07920e3 [draft] e
|
o 980f7dc84c29 [draft] d
|
| o 07f0cc02c068 [draft] c
| |
| o d2ae7f538514 [draft] b
|/
o cb9a9f314b8b [draft] a
$ hg debugstrip 'desc(b)'
saved backup bundle to $TESTTMP/strip-ancestor/.hg/strip-backup/d2ae7f538514-59bd8bc3-backup.hg
The change should be either gone or hidden
$ hg log -G
o 636bc07920e3 [draft] e
|
o 980f7dc84c29 [draft] d
|
o cb9a9f314b8b [draft] a
The backup should not include it (as people tend to manipulate these directly)
$ ls -1 .hg/strip-backup/
d2ae7f538514-59bd8bc3-backup.hg
$ hg debugbundle .hg/strip-backup/*.hg
Stream params: {Compression: BZ}
changegroup -- {nbchanges: 2, version: 03} (mandatory: True)
d2ae7f538514cd87c17547b0de4cea71fe1af9fb
07f0cc02c06869c81ebf33867edef30554020c0d
cache:rev-branch-cache -- {} (mandatory: False)
phase-heads -- {} (mandatory: True)
07f0cc02c06869c81ebf33867edef30554020c0d draft
Shelve should still work
$ hg unshelve
unshelving change 'default'
rebasing shelved changes
$ hg status
A a_file.txt
$ cd ..
strip an unrelated changeset with a lower revnum
------------------------------------------------
$ cp -aR reference-repo strip-unrelated
$ cd strip-unrelated
The internal change is not directly stripped, but it is affected by the strip
and it is in the "temporary backup" zone. The zone that needs to be put in a
temporary bundle while we affect data under it.
$ hg debugstrip 'desc(c)'
saved backup bundle to $TESTTMP/strip-unrelated/.hg/strip-backup/07f0cc02c068-8fd0515f-backup.hg
The change should be either gone or hidden
$ hg log -G
o 636bc07920e3 [draft] e
|
o 980f7dc84c29 [draft] d
|
| o d2ae7f538514 [draft] b
|/
o cb9a9f314b8b [draft] a
The backup should not include it (as people tend to manipulate these directly)
$ ls -1 .hg/strip-backup/
07f0cc02c068-8fd0515f-backup.hg
$ hg debugbundle .hg/strip-backup/*.hg
Stream params: {Compression: BZ}
changegroup -- {nbchanges: 1, version: 03} (mandatory: True)
07f0cc02c06869c81ebf33867edef30554020c0d
cache:rev-branch-cache -- {} (mandatory: False)
phase-heads -- {} (mandatory: True)
07f0cc02c06869c81ebf33867edef30554020c0d draft
Shelve should still work
$ hg unshelve
unshelving change 'default'
rebasing shelved changes
$ hg status
A a_file.txt
$ cd ..
explicitly strip the internal changeset
---------------------------------------
$ cp -aR reference-repo strip-explicit
$ cd strip-explicit
The internal change is directly selected for stripping.
$ hg debugstrip --hidden $shelved_node
The change should be gone
$ hg log -G --hidden
o 636bc07920e3 [draft] e
|
o 980f7dc84c29 [draft] d
|
| o 07f0cc02c068 [draft] c
| |
| o d2ae7f538514 [draft] b
|/
o cb9a9f314b8b [draft] a
We don't need to backup anything
$ ls -1 .hg/strip-backup/
Shelve should still work
$ hg unshelve
unshelving change 'default'
rebasing shelved changes
$ hg status
A a_file.txt
$ cd ..
Explicitly bundling the internal change
=======================================
$ cd reference-repo
try to bundle it alone explicitly
---------------------------------
We should not allow it
$ hg bundle --type v3 --exact --rev $shelved_node --hidden ../internal-01.hg
abort: cannot bundle internal changesets
(1 internal changesets selected)
[255]
$ hg debugbundle ../internal-01.hg
abort: $ENOENT$: '../internal-01.hg'
[255]
try to bundle it with other, somewhat explicitly
------------------------------------------------
We should not allow it
$ hg bundle --type v3 --exact --rev 'desc(b)':: --hidden ../internal-02.hg
abort: cannot bundle internal changesets
(1 internal changesets selected)
[255]
$ hg debugbundle ../internal-02.hg
abort: $ENOENT$: '../internal-02.hg'
[255]
bundle visible ancestors
------------------------
This should succeed as the standard filtering is skipping the internal change naturally
$ hg bundle --type v3 --exact --rev 'desc(b)':: ../internal-03.hg
2 changesets found
$ hg debugbundle ../internal-03.hg
Stream params: {Compression: BZ}
changegroup -- {nbchanges: 2, version: 03} (mandatory: True)
d2ae7f538514cd87c17547b0de4cea71fe1af9fb
07f0cc02c06869c81ebf33867edef30554020c0d
cache:rev-branch-cache -- {} (mandatory: False)
phase-heads -- {} (mandatory: True)
07f0cc02c06869c81ebf33867edef30554020c0d draft
$ cd ..