perf: clear vfs audit_cache before each run
When generating a stream clone, we spend a large amount of time auditing path.
Before this changes, the first run was warming the vfs cache for the other
runs, leading to a large runtime difference and a "faulty" reported timing for
the operation.
We now clear this important cache between run to get a more realistic timing.
Below are some example of median time change when clearing these cases. The
maximum time for a run did not changed significantly.
### data-env-vars.name = mozilla-central-2018-08-01-zstd-sparse-revlog
# benchmark.name = hg.perf.exchange.stream.generate
# bin-env-vars.hg.flavor = default
# bin-env-vars.hg.py-re2-module = default
# benchmark.variants.version = latest
no-clearing: 17.289905
cache-clearing: 21.587965 (+24.86%, +4.30)
## data-env-vars.name = mozilla-central-2024-03-22-zstd-sparse-revlog
no-clearing: 32.670748
cache-clearing: 40.467095 (+23.86%, +7.80)
## data-env-vars.name = mozilla-try-2019-02-18-zstd-sparse-revlog
no-clearing: 37.838858
cache-clearing: 46.072749 (+21.76%, +8.23)
## data-env-vars.name = mozilla-unified-2024-03-22-zstd-sparse-revlog
no-clearing: 32.969395
cache-clearing: 39.646209 (+20.25%, +6.68)
In addition, this significantly reduce the timing difference between the
performance command, from the perf extensions and a `real `hg bundle` call
producing a stream bundle. Some significant differences remain especially on
the "mozilla-try" repositories, but they are now smaller.
Note that some of that difference will actually not be
attributable to the stream generation (like maybe phases or branch map
computation).
Below are some benchmarks done on a currently draft changeset fixing some
unrelated slowness in `hg bundle` (34a78972af409d1ff37c29e60f6ca811ad1a457d)
### data-env-vars.name = mozilla-central-2018-08-01-zstd-sparse-revlog
# bin-env-vars.hg.flavor = default
# bin-env-vars.hg.py-re2-module = default
hg.perf.exchange.stream.generate: 21.587965
hg.command.bundle: 24.301799 (+12.57%, +2.71)
## data-env-vars.name = mozilla-central-2024-03-22-zstd-sparse-revlog
hg.perf.exchange.stream.generate: 40.467095
hg.command.bundle: 44.831317 (+10.78%, +4.36)
## data-env-vars.name = mozilla-unified-2024-03-22-zstd-sparse-revlog
hg.perf.exchange.stream.generate: 39.646209
hg.command.bundle: 45.395258 (+14.50%, +5.75)
## data-env-vars.name = mozilla-try-2019-02-18-zstd-sparse-revlog
hg.perf.exchange.stream.generate: 46.072749
hg.command.bundle: 55.882608 (+21.29%, +9.81)
## data-env-vars.name = mozilla-try-2023-03-22-zlib-general-delta
hg.perf.exchange.stream.generate: 334.716708
hg.command.bundle: 377.856767 (+12.89%, +43.14)
## data-env-vars.name = mozilla-try-2023-03-22-zstd-sparse-revlog
hg.perf.exchange.stream.generate: 302.972301
hg.command.bundle: 326.098755 (+7.63%, +23.13)
===================================================================
Test merge behavior with narrow for item outside of the narrow spec
===================================================================
This test currently check for simple "outside of narrow" merge case. I suspect
there might be more corner case that need testing, so extending this tests, or
replacing it by a more "generative" version, comparing behavior with and without narow.
This the feature is currently working with flat manifest only. This is the only
case tested. Consider using test-case if tree start supporting this case of
merge.
Create some initial setup
$ . "$TESTDIR/narrow-library.sh"
$ hg init server
$ echo root > server/root
$ mkdir server/inside
$ mkdir server/outside
$ echo babar > server/inside/inside-change
$ echo pom > server/outside/outside-changing
$ echo arthur > server/outside/outside-removed
$ hg -R server add server/
adding server/inside/inside-change
adding server/outside/outside-changing
adding server/outside/outside-removed
adding server/root
$ hg -R server commit -m root
$ hg clone ssh://user@dummy/server client --narrow --include inside
requesting all changes
adding changesets
adding manifests
adding file changes
added 1 changesets with 1 changes to 1 files
new changesets a0c415d360e5
updating to branch default
1 files updated, 0 files merged, 0 files removed, 0 files unresolved
"trivial" change outside of narrow spec on the server
$ echo zephir > server/outside/outside-added
$ hg -R server add server/outside/outside-added
$ echo flore > server/outside/outside-changing
$ hg -R server remove server/outside/outside-removed
$ hg -R server commit -m "outside change"
Merge them with some unrelated local change
$ echo celeste > client/inside/inside-change
$ hg -R client commit -m "inside change"
$ hg -R client pull
pulling from ssh://user@dummy/server
searching for changes
adding changesets
adding manifests
adding file changes
added 1 changesets with 0 changes to 0 files (+1 heads)
new changesets f9ec5453023e
(run 'hg heads' to see heads, 'hg merge' to merge)
$ hg -R client merge
0 files updated, 0 files merged, 0 files removed, 0 files unresolved
(branch merge, don't forget to commit)
$ hg -R client ci -m 'merge changes'
$ hg -R client push -r .
pushing to ssh://user@dummy/server
searching for changes
remote: adding changesets
remote: adding manifests
remote: adding file changes
remote: added 2 changesets with 1 changes to 1 files
Checking result
---------------
general sentry of all output
$ hg --repository server manifest --debug --rev 0
360afd990eeff79e4a7f9f3ded5ecd7bc2fd3b59 644 inside/inside-change
7db95ce5cd8e734ad12e3f5f37779a08070a1399 644 outside/outside-changing
1591f6db41a30b68bd94ddccf4a4ce4f4fbe2a44 644 outside/outside-removed
50ecbc31c0e82dd60c2747c434d1f11b85c0e178 644 root
$ hg --repository server manifest --debug --rev 1
360afd990eeff79e4a7f9f3ded5ecd7bc2fd3b59 644 inside/inside-change
486c008d6dddcaeb5e5f99556a121800cdcfb149 644 outside/outside-added
153d7af5e4f53f44475bc0ff2b806c86f019eda4 644 outside/outside-changing
50ecbc31c0e82dd60c2747c434d1f11b85c0e178 644 root
$ hg --repository server manifest --debug --rev 2
1b3ab69c6c847abc8fd25537241fedcd4d188668 644 inside/inside-change
7db95ce5cd8e734ad12e3f5f37779a08070a1399 644 outside/outside-changing
1591f6db41a30b68bd94ddccf4a4ce4f4fbe2a44 644 outside/outside-removed
50ecbc31c0e82dd60c2747c434d1f11b85c0e178 644 root
$ hg --repository server manifest --debug --rev 3
1b3ab69c6c847abc8fd25537241fedcd4d188668 644 inside/inside-change
486c008d6dddcaeb5e5f99556a121800cdcfb149 644 outside/outside-added
153d7af5e4f53f44475bc0ff2b806c86f019eda4 644 outside/outside-changing
50ecbc31c0e82dd60c2747c434d1f11b85c0e178 644 root
The file changed outside should be changed by the merge
$ hg --repository server manifest --debug --rev 'desc("inside change")' | grep outside-changing
7db95ce5cd8e734ad12e3f5f37779a08070a1399 644 outside/outside-changing
$ hg --repository server manifest --debug --rev 'desc("outside change")' | grep outside-changing
153d7af5e4f53f44475bc0ff2b806c86f019eda4 644 outside/outside-changing
$ hg --repository server manifest --debug --rev 'desc("merge")' | grep outside-changing
153d7af5e4f53f44475bc0ff2b806c86f019eda4 644 outside/outside-changing