branchcache: stop writing more branchcache file on disk than needed
Before this change, we were unconditionally writing a branchmap file for the
filter level passed to `update_disk`. This is actually counter productive if no
update were needed for this filter level. In many case, the branch cache for a
filter level is identical to its parent "subset" and it is better to simply
keep the subset update and reuse it every time instead of having to do identical
work for similar subset.
So we change the `update_disk` method to only write a file when that filter
level differ from its parent. This removes many cases where identical files were
written, requiring multiple boring update in the test suite.
The only notable changes is the change to `test-strip-branch-cache.t`, this
case was checking a scenario that no longer reproduce the bug as writing less
branchmap file result in less stalled cache on disk.
Strictly speaking, we could create a more convoluted scenario that create a
similar issue. However the next changeset would also cover that scenario so we
directly updated that test case to a "no longer buggy" state.
--- a/mercurial/branchmap.py Fri Mar 08 16:49:06 2024 +0100
+++ b/mercurial/branchmap.py Sun Mar 10 04:53:17 2024 +0100
@@ -85,7 +85,8 @@
bcache._filtername,
repo.filtername,
)
- bcache.write(repo)
+ if bcache._dirty:
+ bcache.write(repo)
def updatecache(self, repo):
"""Update the cache for the given filtered view on a repository"""
--- a/tests/test-acl.t Fri Mar 08 16:49:06 2024 +0100
+++ b/tests/test-acl.t Sun Mar 10 04:53:17 2024 +0100
@@ -810,7 +810,6 @@
acl: bookmark access granted: "ef1ea85a6374b77d6da9dcda9541f498f2d17df7" on bookmark "moving-bookmark"
bundle2-input-bundle: 7 parts total
updating the branch cache
- invalid branch cache (served.hidden): tip differs
added 1 changesets with 1 changes to 1 files
bundle2-output-bundle: "HG20", 1 parts total
bundle2-output-part: "reply:changegroup" (advisory) (params: 0 advisory) empty payload
@@ -1797,7 +1796,6 @@
bundle2-input-part: total payload size * (glob)
bundle2-input-bundle: 5 parts total
updating the branch cache
- invalid branch cache (served.hidden): tip differs
added 4 changesets with 4 changes to 4 files (+1 heads)
bundle2-output-bundle: "HG20", 1 parts total
bundle2-output-part: "reply:changegroup" (advisory) (params: 0 advisory) empty payload
@@ -2104,7 +2102,6 @@
bundle2-input-part: total payload size * (glob)
bundle2-input-bundle: 5 parts total
updating the branch cache
- invalid branch cache (served.hidden): tip differs
added 4 changesets with 4 changes to 4 files (+1 heads)
bundle2-output-bundle: "HG20", 1 parts total
bundle2-output-part: "reply:changegroup" (advisory) (params: 0 advisory) empty payload
@@ -2196,7 +2193,6 @@
bundle2-input-part: total payload size * (glob)
bundle2-input-bundle: 5 parts total
updating the branch cache
- invalid branch cache (served.hidden): tip differs
added 4 changesets with 4 changes to 4 files (+1 heads)
bundle2-output-bundle: "HG20", 1 parts total
bundle2-output-part: "reply:changegroup" (advisory) (params: 0 advisory) empty payload
@@ -2360,7 +2356,6 @@
bundle2-input-part: total payload size * (glob)
bundle2-input-bundle: 5 parts total
updating the branch cache
- invalid branch cache (served.hidden): tip differs
added 4 changesets with 4 changes to 4 files (+1 heads)
bundle2-output-bundle: "HG20", 1 parts total
bundle2-output-part: "reply:changegroup" (advisory) (params: 0 advisory) empty payload
--- a/tests/test-blackbox.t Fri Mar 08 16:49:06 2024 +0100
+++ b/tests/test-blackbox.t Sun Mar 10 04:53:17 2024 +0100
@@ -127,13 +127,11 @@
added 1 changesets with 1 changes to 1 files
new changesets d02f48003e62
(run 'hg update' to get a working copy)
- $ hg blackbox -l 6
+ $ hg blackbox -l 4
1970-01-01 00:00:00.000 bob @6563da9dcf87b1949716e38ff3e3dfaa3198eb06 (5000)> wrote branch cache (served) with 1 labels and 2 nodes
- 1970-01-01 00:00:00.000 bob @6563da9dcf87b1949716e38ff3e3dfaa3198eb06 (5000)> updated branch cache (served.hidden) in * seconds (glob)
- 1970-01-01 00:00:00.000 bob @6563da9dcf87b1949716e38ff3e3dfaa3198eb06 (5000)> wrote branch cache (served.hidden) with 1 labels and 2 nodes
1970-01-01 00:00:00.000 bob @6563da9dcf87b1949716e38ff3e3dfaa3198eb06 (5000)> 1 incoming changes - new heads: d02f48003e62
1970-01-01 00:00:00.000 bob @6563da9dcf87b1949716e38ff3e3dfaa3198eb06 (5000)> pull exited 0 after * seconds (glob)
- 1970-01-01 00:00:00.000 bob @6563da9dcf87b1949716e38ff3e3dfaa3198eb06 (5000)> blackbox -l 6
+ 1970-01-01 00:00:00.000 bob @6563da9dcf87b1949716e38ff3e3dfaa3198eb06 (5000)> blackbox -l 4
we must not cause a failure if we cannot write to the log
@@ -193,8 +191,8 @@
$ hg blackbox -l 6
1970-01-01 00:00:00.000 bob @73f6ee326b27d820b0472f1a825e3a50f3dc489b (5000)> strip tip
1970-01-01 00:00:00.000 bob @6563da9dcf87b1949716e38ff3e3dfaa3198eb06 (5000)> saved backup bundle to $TESTTMP/blackboxtest2/.hg/strip-backup/73f6ee326b27-7612e004-backup.hg
- 1970-01-01 00:00:00.000 bob @6563da9dcf87b1949716e38ff3e3dfaa3198eb06 (5000)> updated branch cache (immutable) in * seconds (glob)
- 1970-01-01 00:00:00.000 bob @6563da9dcf87b1949716e38ff3e3dfaa3198eb06 (5000)> wrote branch cache (immutable) with 1 labels and 2 nodes
+ 1970-01-01 00:00:00.000 bob @6563da9dcf87b1949716e38ff3e3dfaa3198eb06 (5000)> updated branch cache (base) in * seconds (glob)
+ 1970-01-01 00:00:00.000 bob @6563da9dcf87b1949716e38ff3e3dfaa3198eb06 (5000)> wrote branch cache (base) with 1 labels and 2 nodes
1970-01-01 00:00:00.000 bob @6563da9dcf87b1949716e38ff3e3dfaa3198eb06 (5000)> strip tip exited 0 after * seconds (glob)
1970-01-01 00:00:00.000 bob @6563da9dcf87b1949716e38ff3e3dfaa3198eb06 (5000)> blackbox -l 6
--- a/tests/test-branches.t Fri Mar 08 16:49:06 2024 +0100
+++ b/tests/test-branches.t Sun Mar 10 04:53:17 2024 +0100
@@ -1316,7 +1316,7 @@
new changesets 2ab8003a1750:99ba08759bc7
updating to branch A
0 files updated, 0 files merged, 0 files removed, 0 files unresolved
- $ cat branchmap-update-01/.hg/cache/branch2-served
+ $ cat branchmap-update-01/.hg/cache/branch2-base
99ba08759bc7f6fdbe5304e83d0387f35c082479 1
99ba08759bc7f6fdbe5304e83d0387f35c082479 o A
$ hg -R branchmap-update-01 unbundle bundle.hg
@@ -1350,7 +1350,7 @@
updating to branch A
0 files updated, 0 files merged, 0 files removed, 0 files unresolved
- $ cat branchmap-update-02/.hg/cache/branch2-served
+ $ cat branchmap-update-02/.hg/cache/branch2-base
99ba08759bc7f6fdbe5304e83d0387f35c082479 1
99ba08759bc7f6fdbe5304e83d0387f35c082479 o A
$ hg -R branchmap-update-02 unbundle bundle.hg --config "hooks.pretxnclose=python:$TESTTMP/simplehook.py:hook"
@@ -1361,6 +1361,6 @@
rollback completed
abort: pretxnclose hook failed
[40]
- $ cat branchmap-update-02/.hg/cache/branch2-served
+ $ cat branchmap-update-02/.hg/cache/branch2-base
99ba08759bc7f6fdbe5304e83d0387f35c082479 1
99ba08759bc7f6fdbe5304e83d0387f35c082479 o A
--- a/tests/test-clone-stream.t Fri Mar 08 16:49:06 2024 +0100
+++ b/tests/test-clone-stream.t Sun Mar 10 04:53:17 2024 +0100
@@ -220,11 +220,7 @@
#if no-stream-legacy
$ ls -1 clone1/.hg/cache
branch2-base
- branch2-immutable
branch2-served
- branch2-served.hidden
- branch2-visible
- branch2-visible-hidden
rbc-names-v1
rbc-revs-v1
tags2
--- a/tests/test-clone.t Fri Mar 08 16:49:06 2024 +0100
+++ b/tests/test-clone.t Sun Mar 10 04:53:17 2024 +0100
@@ -47,11 +47,7 @@
$ ls .hg/cache
branch2-base
- branch2-immutable
branch2-served
- branch2-served.hidden
- branch2-visible
- branch2-visible-hidden
rbc-names-v1
rbc-revs-v1
tags2
@@ -71,42 +67,34 @@
#if hardlink
$ hg --debug clone -U . ../c --config progress.debug=true
- linking: 1/16 files (6.25%) (no-rust !)
- linking: 2/16 files (12.50%) (no-rust !)
- linking: 3/16 files (18.75%) (no-rust !)
- linking: 4/16 files (25.00%) (no-rust !)
- linking: 5/16 files (31.25%) (no-rust !)
- linking: 6/16 files (37.50%) (no-rust !)
- linking: 7/16 files (43.75%) (no-rust !)
- linking: 8/16 files (50.00%) (no-rust !)
- linking: 9/16 files (56.25%) (no-rust !)
- linking: 10/16 files (62.50%) (no-rust !)
- linking: 11/16 files (68.75%) (no-rust !)
- linking: 12/16 files (75.00%) (no-rust !)
- linking: 13/16 files (81.25%) (no-rust !)
- linking: 14/16 files (87.50%) (no-rust !)
- linking: 15/16 files (93.75%) (no-rust !)
- linking: 16/16 files (100.00%) (no-rust !)
- linked 16 files (no-rust !)
- linking: 1/18 files (5.56%) (rust !)
- linking: 2/18 files (11.11%) (rust !)
- linking: 3/18 files (16.67%) (rust !)
- linking: 4/18 files (22.22%) (rust !)
- linking: 5/18 files (27.78%) (rust !)
- linking: 6/18 files (33.33%) (rust !)
- linking: 7/18 files (38.89%) (rust !)
- linking: 8/18 files (44.44%) (rust !)
- linking: 9/18 files (50.00%) (rust !)
- linking: 10/18 files (55.56%) (rust !)
- linking: 11/18 files (61.11%) (rust !)
- linking: 12/18 files (66.67%) (rust !)
- linking: 13/18 files (72.22%) (rust !)
- linking: 14/18 files (77.78%) (rust !)
- linking: 15/18 files (83.33%) (rust !)
- linking: 16/18 files (88.89%) (rust !)
- linking: 17/18 files (94.44%) (rust !)
- linking: 18/18 files (100.00%) (rust !)
- linked 18 files (rust !)
+ linking: 1/12 files (8.33%) (no-rust !)
+ linking: 2/12 files (16.67%) (no-rust !)
+ linking: 3/12 files (25.00%) (no-rust !)
+ linking: 4/12 files (33.33%) (no-rust !)
+ linking: 5/12 files (41.67%) (no-rust !)
+ linking: 6/12 files (50.00%) (no-rust !)
+ linking: 7/12 files (58.33%) (no-rust !)
+ linking: 8/12 files (66.67%) (no-rust !)
+ linking: 9/12 files (75.00%) (no-rust !)
+ linking: 10/12 files (83.33%) (no-rust !)
+ linking: 11/12 files (91.67%) (no-rust !)
+ linking: 12/12 files (100.00%) (no-rust !)
+ linked 12 files (no-rust !)
+ linking: 1/14 files (7.14%) (rust !)
+ linking: 2/14 files (14.29%) (rust !)
+ linking: 3/14 files (21.43%) (rust !)
+ linking: 4/14 files (28.57%) (rust !)
+ linking: 5/14 files (35.71%) (rust !)
+ linking: 6/14 files (42.86%) (rust !)
+ linking: 7/14 files (50.00%) (rust !)
+ linking: 8/14 files (57.14%) (rust !)
+ linking: 9/14 files (64.29%) (rust !)
+ linking: 10/14 files (71.43%) (rust !)
+ linking: 11/14 files (78.57%) (rust !)
+ linking: 12/14 files (85.71%) (rust !)
+ linking: 13/14 files (92.86%) (rust !)
+ linking: 14/14 files (100.00%) (rust !)
+ linked 14 files (rust !)
updating the branch cache
#else
$ hg --debug clone -U . ../c --config progress.debug=true
@@ -125,11 +113,7 @@
$ ls .hg/cache
branch2-base
- branch2-immutable
branch2-served
- branch2-served.hidden
- branch2-visible
- branch2-visible-hidden
rbc-names-v1
rbc-revs-v1
tags2
--- a/tests/test-debugcommands.t Fri Mar 08 16:49:06 2024 +0100
+++ b/tests/test-debugcommands.t Sun Mar 10 04:53:17 2024 +0100
@@ -652,12 +652,7 @@
.hg/cache/rbc-revs-v1
.hg/cache/rbc-names-v1
.hg/cache/hgtagsfnodes1
- .hg/cache/branch2-visible-hidden
- .hg/cache/branch2-visible
- .hg/cache/branch2-served.hidden
.hg/cache/branch2-served
- .hg/cache/branch2-immutable
- .hg/cache/branch2-base
Test debug::unbundle
--- a/tests/test-hardlinks.t Fri Mar 08 16:49:06 2024 +0100
+++ b/tests/test-hardlinks.t Sun Mar 10 04:53:17 2024 +0100
@@ -263,11 +263,7 @@
2 r4/.hg/00changelog.i
[24] r4/.hg/branch (re)
2 r4/.hg/cache/branch2-base
- 2 r4/.hg/cache/branch2-immutable
2 r4/.hg/cache/branch2-served
- 2 r4/.hg/cache/branch2-served.hidden
- 2 r4/.hg/cache/branch2-visible
- 2 r4/.hg/cache/branch2-visible-hidden
2 r4/.hg/cache/rbc-names-v1
2 r4/.hg/cache/rbc-revs-v1
2 r4/.hg/cache/tags2
@@ -320,11 +316,7 @@
2 r4/.hg/00changelog.i
1 r4/.hg/branch
2 r4/.hg/cache/branch2-base
- 2 r4/.hg/cache/branch2-immutable
2 r4/.hg/cache/branch2-served
- 2 r4/.hg/cache/branch2-served.hidden
- 2 r4/.hg/cache/branch2-visible
- 2 r4/.hg/cache/branch2-visible-hidden
2 r4/.hg/cache/rbc-names-v1
2 r4/.hg/cache/rbc-revs-v1
2 r4/.hg/cache/tags2
--- a/tests/test-server-view.t Fri Mar 08 16:49:06 2024 +0100
+++ b/tests/test-server-view.t Sun Mar 10 04:53:17 2024 +0100
@@ -36,12 +36,7 @@
$ hg -R test --config experimental.extra-filter-revs='not public()' debugupdatecache
$ ls -1 test/.hg/cache/
branch2-base%89c45d2fa07e
- branch2-immutable%89c45d2fa07e
branch2-served
- branch2-served%89c45d2fa07e
- branch2-served.hidden%89c45d2fa07e
- branch2-visible%89c45d2fa07e
- branch2-visible-hidden%89c45d2fa07e
hgtagsfnodes1
rbc-names-v1
rbc-revs-v1
--- a/tests/test-share.t Fri Mar 08 16:49:06 2024 +0100
+++ b/tests/test-share.t Sun Mar 10 04:53:17 2024 +0100
@@ -63,11 +63,7 @@
1 files updated, 0 files merged, 0 files removed, 0 files unresolved
$ ls -1 ../repo2-clone/.hg/cache
branch2-base
- branch2-immutable
branch2-served
- branch2-served.hidden
- branch2-visible
- branch2-visible-hidden
rbc-names-v1
rbc-revs-v1
tags2
--- a/tests/test-ssh.t Fri Mar 08 16:49:06 2024 +0100
+++ b/tests/test-ssh.t Sun Mar 10 04:53:17 2024 +0100
@@ -72,8 +72,8 @@
$ hg -R local-stream book mybook
$ hg clone --stream ssh://user@dummy/local-stream stream2
streaming all changes
- 16 files to transfer, * of data (glob) (no-rust !)
- 18 files to transfer, * of data (glob) (rust !)
+ 12 files to transfer, * of data (glob) (no-rust !)
+ 14 files to transfer, * of data (glob) (rust !)
transferred * in * seconds (*) (glob)
updating to branch default
2 files updated, 0 files merged, 0 files removed, 0 files unresolved
--- a/tests/test-strip-branch-cache.t Fri Mar 08 16:49:06 2024 +0100
+++ b/tests/test-strip-branch-cache.t Sun Mar 10 04:53:17 2024 +0100
@@ -1,3 +1,5 @@
+This test cover a bug that no longer exist.
+
Define helpers.
$ hg_log () { hg log -G -T "{rev}:{node|short}"; }
@@ -18,7 +20,10 @@
$ hg pull -q ../repo
- $ cat .hg/cache/branch2-visible
+ $ ls -1 .hg/cache/branch?*
+ .hg/cache/branch2-base
+ .hg/cache/branch2-served
+ $ cat .hg/cache/branch?-served
222ae9789a75703f9836e44de7db179cbfd420ee 2
a3498d6e39376d2456425dd8c692367bdbf00fa2 o default
222ae9789a75703f9836e44de7db179cbfd420ee o default
@@ -33,24 +38,38 @@
$ strip '1:'
-The branchmap cache is not adjusted on strip.
-Now mentions a changelog entry that has been stripped.
+After the strip the "served" cache is now identical to the "base" one, and the
+older one have been actively deleted.
- $ cat .hg/cache/branch2-visible
+ $ ls -1 .hg/cache/branch?*
+ .hg/cache/branch2-base
+ .hg/cache/branch2-served
+ $ cat .hg/cache/branch?-served
222ae9789a75703f9836e44de7db179cbfd420ee 2
a3498d6e39376d2456425dd8c692367bdbf00fa2 o default
222ae9789a75703f9836e44de7db179cbfd420ee o default
- $ commit c
-
-Not adjusted on commit, either.
+We do a new commit and we get a new valid branchmap for the served version
- $ cat .hg/cache/branch2-visible
- 222ae9789a75703f9836e44de7db179cbfd420ee 2
- a3498d6e39376d2456425dd8c692367bdbf00fa2 o default
- 222ae9789a75703f9836e44de7db179cbfd420ee o default
+ $ commit c
+ $ ls -1 .hg/cache/branch?*
+ .hg/cache/branch2-base
+ .hg/cache/branch2-served
+ $ cat .hg/cache/branch?-served
+ a1602b357cfca067600406eb19060c7128804d72 1
+ a1602b357cfca067600406eb19060c7128804d72 o default
+
On pull we end up with the same tip, and so wrongly reuse the invalid cache and crash.
- $ hg pull ../repo 2>&1 | grep 'ValueError:'
- ValueError: node a3498d6e39376d2456425dd8c692367bdbf00fa2 does not exist (known-bad-output !)
+ $ hg pull ../repo --quiet
+ $ hg heads -T '{rev} {node} {branch}\n'
+ 2 222ae9789a75703f9836e44de7db179cbfd420ee default
+ 1 a1602b357cfca067600406eb19060c7128804d72 default
+ $ ls -1 .hg/cache/branch?*
+ .hg/cache/branch2-base
+ .hg/cache/branch2-served
+ $ cat .hg/cache/branch?-served
+ 222ae9789a75703f9836e44de7db179cbfd420ee 2
+ a1602b357cfca067600406eb19060c7128804d72 o default
+ 222ae9789a75703f9836e44de7db179cbfd420ee o default
--- a/tests/test-tags.t Fri Mar 08 16:49:06 2024 +0100
+++ b/tests/test-tags.t Sun Mar 10 04:53:17 2024 +0100
@@ -792,11 +792,6 @@
$ ls tagsclient/.hg/cache
branch2-base
- branch2-immutable
- branch2-served
- branch2-served.hidden
- branch2-visible
- branch2-visible-hidden
hgtagsfnodes1
rbc-names-v1
rbc-revs-v1
@@ -823,11 +818,6 @@
$ ls tagsclient/.hg/cache
branch2-base
- branch2-immutable
- branch2-served
- branch2-served.hidden
- branch2-visible
- branch2-visible-hidden
hgtagsfnodes1
rbc-names-v1
rbc-revs-v1
--- a/tests/test-treemanifest.t Fri Mar 08 16:49:06 2024 +0100
+++ b/tests/test-treemanifest.t Sun Mar 10 04:53:17 2024 +0100
@@ -761,8 +761,8 @@
$ hg clone --config experimental.changegroup3=True --stream -U \
> http://localhost:$HGPORT1 stream-clone-basicstore
streaming all changes
- 29 files to transfer, * of data (glob) (no-rust !)
- 31 files to transfer, * of data (glob) (rust !)
+ 24 files to transfer, * of data (glob) (no-rust !)
+ 26 files to transfer, * of data (glob) (rust !)
transferred * in * seconds (*) (glob)
$ hg -R stream-clone-basicstore verify -q
$ cat port-1-errors.log
@@ -771,8 +771,8 @@
$ hg clone --config experimental.changegroup3=True --stream -U \
> http://localhost:$HGPORT2 stream-clone-encodedstore
streaming all changes
- 29 files to transfer, * of data (glob) (no-rust !)
- 31 files to transfer, * of data (glob) (rust !)
+ 24 files to transfer, * of data (glob) (no-rust !)
+ 26 files to transfer, * of data (glob) (rust !)
transferred * in * seconds (*) (glob)
$ hg -R stream-clone-encodedstore verify -q
$ cat port-2-errors.log