perf: clear vfs audit_cache before each run
When generating a stream clone, we spend a large amount of time auditing path.
Before this changes, the first run was warming the vfs cache for the other
runs, leading to a large runtime difference and a "faulty" reported timing for
the operation.
We now clear this important cache between run to get a more realistic timing.
Below are some example of median time change when clearing these cases. The
maximum time for a run did not changed significantly.
### data-env-vars.name = mozilla-central-2018-08-01-zstd-sparse-revlog
# benchmark.name = hg.perf.exchange.stream.generate
# bin-env-vars.hg.flavor = default
# bin-env-vars.hg.py-re2-module = default
# benchmark.variants.version = latest
no-clearing: 17.289905
cache-clearing: 21.587965 (+24.86%, +4.30)
## data-env-vars.name = mozilla-central-2024-03-22-zstd-sparse-revlog
no-clearing: 32.670748
cache-clearing: 40.467095 (+23.86%, +7.80)
## data-env-vars.name = mozilla-try-2019-02-18-zstd-sparse-revlog
no-clearing: 37.838858
cache-clearing: 46.072749 (+21.76%, +8.23)
## data-env-vars.name = mozilla-unified-2024-03-22-zstd-sparse-revlog
no-clearing: 32.969395
cache-clearing: 39.646209 (+20.25%, +6.68)
In addition, this significantly reduce the timing difference between the
performance command, from the perf extensions and a `real `hg bundle` call
producing a stream bundle. Some significant differences remain especially on
the "mozilla-try" repositories, but they are now smaller.
Note that some of that difference will actually not be
attributable to the stream generation (like maybe phases or branch map
computation).
Below are some benchmarks done on a currently draft changeset fixing some
unrelated slowness in `hg bundle` (34a78972af409d1ff37c29e60f6ca811ad1a457d)
### data-env-vars.name = mozilla-central-2018-08-01-zstd-sparse-revlog
# bin-env-vars.hg.flavor = default
# bin-env-vars.hg.py-re2-module = default
hg.perf.exchange.stream.generate: 21.587965
hg.command.bundle: 24.301799 (+12.57%, +2.71)
## data-env-vars.name = mozilla-central-2024-03-22-zstd-sparse-revlog
hg.perf.exchange.stream.generate: 40.467095
hg.command.bundle: 44.831317 (+10.78%, +4.36)
## data-env-vars.name = mozilla-unified-2024-03-22-zstd-sparse-revlog
hg.perf.exchange.stream.generate: 39.646209
hg.command.bundle: 45.395258 (+14.50%, +5.75)
## data-env-vars.name = mozilla-try-2019-02-18-zstd-sparse-revlog
hg.perf.exchange.stream.generate: 46.072749
hg.command.bundle: 55.882608 (+21.29%, +9.81)
## data-env-vars.name = mozilla-try-2023-03-22-zlib-general-delta
hg.perf.exchange.stream.generate: 334.716708
hg.command.bundle: 377.856767 (+12.89%, +43.14)
## data-env-vars.name = mozilla-try-2023-03-22-zstd-sparse-revlog
hg.perf.exchange.stream.generate: 302.972301
hg.command.bundle: 326.098755 (+7.63%, +23.13)
$ echo "[extensions]" >> $HGRCPATH
$ echo "mq=" >> $HGRCPATH
make a test repository that looks like this:
o 2:28bc7b1afd6a
|
| @ 1:d7fe2034f71b
|/
o 0/62ecad8b70e5
$ hg init r0
$ cd r0
$ touch f0
$ hg ci -m0 -Aq
$ touch f1
$ hg ci -m1 -Aq
$ hg update 0 -q
$ touch f2
$ hg ci -m2 -Aq
$ hg update 1 -q
make some patches with a parent: 1:d7fe2034f71b -> p0 -> p1
$ echo cp0 >> fp0
$ hg add fp0
$ hg ci -m p0 -d "0 0"
$ hg export -r. > p0
$ hg strip -qn .
$ hg qimport p0
adding p0 to series file
$ hg qpush
applying p0
now at: p0
$ echo cp1 >> fp1
$ hg add fp1
$ hg qnew p1 -d "0 0"
$ hg qpop -aq
patch queue now empty
qpush --exact when at the parent
$ hg update 1 -q
$ hg qpush -e
applying p0
now at: p0
$ hg parents -qr qbase
1:d7fe2034f71b
$ hg qpop -aq
patch queue now empty
$ hg qpush -e p0
applying p0
now at: p0
$ hg parents -qr qbase
1:d7fe2034f71b
$ hg qpop -aq
patch queue now empty
$ hg qpush -e p1
applying p0
applying p1
now at: p1
$ hg parents -qr qbase
1:d7fe2034f71b
$ hg qpop -aq
patch queue now empty
qpush --exact when at another rev
$ hg update 0 -q
$ hg qpush -e
1 files updated, 0 files merged, 0 files removed, 0 files unresolved
applying p0
now at: p0
$ hg parents -qr qbase
1:d7fe2034f71b
$ hg qpop -aq
patch queue now empty
$ hg update 0 -q
$ hg qpush -e p0
1 files updated, 0 files merged, 0 files removed, 0 files unresolved
applying p0
now at: p0
$ hg parents -qr qbase
1:d7fe2034f71b
$ hg qpop -aq
patch queue now empty
$ hg update 0 -q
$ hg qpush -e p1
1 files updated, 0 files merged, 0 files removed, 0 files unresolved
applying p0
applying p1
now at: p1
$ hg parents -qr qbase
1:d7fe2034f71b
$ hg qpop -aq
patch queue now empty
$ hg update 0 -q
$ hg qpush -ea
1 files updated, 0 files merged, 0 files removed, 0 files unresolved
applying p0
applying p1
now at: p1
$ hg parents -qr qbase
1:d7fe2034f71b
$ hg qpop -aq
patch queue now empty
qpush --exact while crossing branches
$ hg update 2 -q
$ hg qpush -e
1 files updated, 0 files merged, 1 files removed, 0 files unresolved
applying p0
now at: p0
$ hg parents -qr qbase
1:d7fe2034f71b
$ hg qpop -aq
patch queue now empty
$ hg update 2 -q
$ hg qpush -e p0
1 files updated, 0 files merged, 1 files removed, 0 files unresolved
applying p0
now at: p0
$ hg parents -qr qbase
1:d7fe2034f71b
$ hg qpop -aq
patch queue now empty
$ hg update 2 -q
$ hg qpush -e p1
1 files updated, 0 files merged, 1 files removed, 0 files unresolved
applying p0
applying p1
now at: p1
$ hg parents -qr qbase
1:d7fe2034f71b
$ hg qpop -aq
patch queue now empty
$ hg update 2 -q
$ hg qpush -ea
1 files updated, 0 files merged, 1 files removed, 0 files unresolved
applying p0
applying p1
now at: p1
$ hg parents -qr qbase
1:d7fe2034f71b
$ hg qpop -aq
patch queue now empty
qpush --exact --force with changes to an unpatched file
$ hg update 1 -q
$ echo c0 >> f0
$ hg qpush -e
abort: local changes found
[255]
$ hg qpush -ef
applying p0
now at: p0
$ cat f0
c0
$ rm f0
$ touch f0
$ hg qpop -aq
patch queue now empty
$ hg update 1 -q
$ echo c0 >> f0
$ hg qpush -e p1
abort: local changes found
[255]
$ hg qpush -e p1 -f
applying p0
applying p1
now at: p1
$ cat f0
c0
$ rm f0
$ touch f0
$ hg qpop -aq
patch queue now empty
qpush --exact --force with changes to a patched file
$ hg update 1 -q
$ echo cp0-bad >> fp0
$ hg add fp0
$ hg qpush -e
abort: local changes found
[255]
$ hg qpush -ef
applying p0
file fp0 already exists
1 out of 1 hunks FAILED -- saving rejects to file fp0.rej
patch failed, unable to continue (try -v)
patch failed, rejects left in working directory
errors during apply, please fix and qrefresh p0
[2]
$ cat fp0
cp0-bad
$ cat fp0.rej
--- fp0
+++ fp0
@@ -0,0 +1,1 @@
+cp0
$ hg qpop -aqf
patch queue now empty
$ rm fp0
$ rm fp0.rej
$ hg update 1 -q
$ echo cp1-bad >> fp1
$ hg add fp1
$ hg qpush -e p1
abort: local changes found
[255]
$ hg qpush -e p1 -f
applying p0
applying p1
file fp1 already exists
1 out of 1 hunks FAILED -- saving rejects to file fp1.rej
patch failed, unable to continue (try -v)
patch failed, rejects left in working directory
errors during apply, please fix and qrefresh p1
[2]
$ cat fp1
cp1-bad
$ cat fp1.rej
--- fp1
+++ fp1
@@ -0,0 +1,1 @@
+cp1
$ hg qpop -aqf
patch queue now empty
$ hg forget fp1
$ rm fp1
$ rm fp1.rej
qpush --exact when already at a patch
$ hg update 1
0 files updated, 0 files merged, 0 files removed, 0 files unresolved
$ hg qpush -e p0
applying p0
now at: p0
$ hg qpush -e p1
abort: cannot push --exact with applied patches
[255]
$ hg qpop -aq
patch queue now empty
qpush --exact --move should fail
$ hg qpush -e --move p1
abort: cannot use --exact and --move together
[255]
qpush --exact a patch without a parent recorded
$ hg qpush -q
now at: p0
$ grep -v '# Parent' .hg/patches/p0 > p0.new
$ mv p0.new .hg/patches/p0
$ hg qpop -aq
patch queue now empty
$ hg qpush -e
abort: p0 does not have a parent recorded
[255]
$ hg qpush -e p0
abort: p0 does not have a parent recorded
[255]
$ hg qpush -e p1
abort: p0 does not have a parent recorded
[255]
$ hg qpush -ea
abort: p0 does not have a parent recorded
[255]
$ cd ..