perf: clear vfs audit_cache before each run
When generating a stream clone, we spend a large amount of time auditing path.
Before this changes, the first run was warming the vfs cache for the other
runs, leading to a large runtime difference and a "faulty" reported timing for
the operation.
We now clear this important cache between run to get a more realistic timing.
Below are some example of median time change when clearing these cases. The
maximum time for a run did not changed significantly.
### data-env-vars.name = mozilla-central-2018-08-01-zstd-sparse-revlog
# benchmark.name = hg.perf.exchange.stream.generate
# bin-env-vars.hg.flavor = default
# bin-env-vars.hg.py-re2-module = default
# benchmark.variants.version = latest
no-clearing: 17.289905
cache-clearing: 21.587965 (+24.86%, +4.30)
## data-env-vars.name = mozilla-central-2024-03-22-zstd-sparse-revlog
no-clearing: 32.670748
cache-clearing: 40.467095 (+23.86%, +7.80)
## data-env-vars.name = mozilla-try-2019-02-18-zstd-sparse-revlog
no-clearing: 37.838858
cache-clearing: 46.072749 (+21.76%, +8.23)
## data-env-vars.name = mozilla-unified-2024-03-22-zstd-sparse-revlog
no-clearing: 32.969395
cache-clearing: 39.646209 (+20.25%, +6.68)
In addition, this significantly reduce the timing difference between the
performance command, from the perf extensions and a `real `hg bundle` call
producing a stream bundle. Some significant differences remain especially on
the "mozilla-try" repositories, but they are now smaller.
Note that some of that difference will actually not be
attributable to the stream generation (like maybe phases or branch map
computation).
Below are some benchmarks done on a currently draft changeset fixing some
unrelated slowness in `hg bundle` (34a78972af409d1ff37c29e60f6ca811ad1a457d)
### data-env-vars.name = mozilla-central-2018-08-01-zstd-sparse-revlog
# bin-env-vars.hg.flavor = default
# bin-env-vars.hg.py-re2-module = default
hg.perf.exchange.stream.generate: 21.587965
hg.command.bundle: 24.301799 (+12.57%, +2.71)
## data-env-vars.name = mozilla-central-2024-03-22-zstd-sparse-revlog
hg.perf.exchange.stream.generate: 40.467095
hg.command.bundle: 44.831317 (+10.78%, +4.36)
## data-env-vars.name = mozilla-unified-2024-03-22-zstd-sparse-revlog
hg.perf.exchange.stream.generate: 39.646209
hg.command.bundle: 45.395258 (+14.50%, +5.75)
## data-env-vars.name = mozilla-try-2019-02-18-zstd-sparse-revlog
hg.perf.exchange.stream.generate: 46.072749
hg.command.bundle: 55.882608 (+21.29%, +9.81)
## data-env-vars.name = mozilla-try-2023-03-22-zlib-general-delta
hg.perf.exchange.stream.generate: 334.716708
hg.command.bundle: 377.856767 (+12.89%, +43.14)
## data-env-vars.name = mozilla-try-2023-03-22-zstd-sparse-revlog
hg.perf.exchange.stream.generate: 302.972301
hg.command.bundle: 326.098755 (+7.63%, +23.13)
$ cat <<EOF >> $HGRCPATH
> [extensions]
> notify=
>
> [hooks]
> changegroup.notify = python:hgext.notify.hook
>
> [notify]
> sources = push
> diffstat = False
> maxsubject = 10
>
> [usersubs]
> foo@bar = *
>
> [reposubs]
> * = baz
> EOF
$ hg init a
clone
$ hg --traceback clone a b
updating to branch default
0 files updated, 0 files merged, 0 files removed, 0 files unresolved
$ echo a > b/a
commit
$ hg --traceback --cwd b commit -Ama
adding a
$ echo a >> b/a
commit
$ hg --traceback --cwd b commit -Amb
push
$ hg --traceback --cwd b push ../a 2>&1 |
> "$PYTHON" $TESTDIR/unwrap-message-id.py | \
> "$PYTHON" -c 'import sys,re; print(re.sub("\n\t", " ", sys.stdin.read()), end="")'
pushing to ../a
searching for changes
adding changesets
adding manifests
adding file changes
added 2 changesets with 2 changes to 1 files
MIME-Version: 1.0
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: 7bit
Date: * (glob)
Subject: * (glob)
From: test
X-Hg-Notification: changeset cb9a9f314b8b
Message-Id: <*> (glob)
To: baz, foo@bar
changeset cb9a9f314b8b in $TESTTMP/a
details: $TESTTMP/a?cmd=changeset;node=cb9a9f314b8b
summary: a
changeset ba677d0156c1 in $TESTTMP/a
details: $TESTTMP/a?cmd=changeset;node=ba677d0156c1
summary: b
diffs (6 lines):
diff -r 000000000000 -r ba677d0156c1 a
--- /dev/null Thu Jan 01 00:00:00 1970 +0000
+++ b/a Thu Jan 01 00:00:00 1970 +0000
@@ -0,0 +1,2 @@
+a
+a
$ hg --cwd a rollback
repository tip rolled back to revision -1 (undo push)
unbundle with unrelated source
$ hg --cwd b bundle ../test.hg ../a
searching for changes
2 changesets found
$ hg --cwd a unbundle ../test.hg
adding changesets
adding manifests
adding file changes
added 2 changesets with 2 changes to 1 files
new changesets cb9a9f314b8b:ba677d0156c1 (2 drafts)
(run 'hg update' to get a working copy)
$ hg --cwd a rollback
repository tip rolled back to revision -1 (undo unbundle)
unbundle with correct source
$ hg --config notify.sources=unbundle --cwd a unbundle ../test.hg 2>&1 |
> "$PYTHON" $TESTDIR/unwrap-message-id.py | \
> "$PYTHON" -c 'import sys,re; print(re.sub("\n\t", " ", sys.stdin.read()), end="")'
adding changesets
adding manifests
adding file changes
added 2 changesets with 2 changes to 1 files
new changesets cb9a9f314b8b:ba677d0156c1 (2 drafts)
MIME-Version: 1.0
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: 7bit
Date: * (glob)
Subject: * (glob)
From: test
X-Hg-Notification: changeset cb9a9f314b8b
Message-Id: <*> (glob)
To: baz, foo@bar
changeset cb9a9f314b8b in $TESTTMP/a
details: $TESTTMP/a?cmd=changeset;node=cb9a9f314b8b
summary: a
changeset ba677d0156c1 in $TESTTMP/a
details: $TESTTMP/a?cmd=changeset;node=ba677d0156c1
summary: b
diffs (6 lines):
diff -r 000000000000 -r ba677d0156c1 a
--- /dev/null Thu Jan 01 00:00:00 1970 +0000
+++ b/a Thu Jan 01 00:00:00 1970 +0000
@@ -0,0 +1,2 @@
+a
+a
(run 'hg update' to get a working copy)
Check that using the first committer as the author of a changeset works:
Check that the config option works.
Check that the first committer is indeed used for "From:".
Check that the merge user is NOT used for "From:"
Create new file
$ echo a > b/b
$ echo b >> b/b
$ echo c >> b/b
$ hg --traceback --cwd b commit -Amnewfile -u committer_1
adding b
commit as one user
$ echo x > b/b
$ echo b >> b/b
$ echo c >> b/b
$ hg --traceback --cwd b commit -Amx -u committer_2
commit as other user, change file so we can do an (automatic) merge
$ hg --cwd b up 2
1 files updated, 0 files merged, 0 files removed, 0 files unresolved
$ echo a > b/b
$ echo b >> b/b
$ echo y >> b/b
$ hg --traceback --cwd b commit -Amy -u committer_3
created new head
merge as a different user
$ hg --cwd b merge --config notify.fromauthor=True
merging b
0 files updated, 1 files merged, 0 files removed, 0 files unresolved
(branch merge, don't forget to commit)
$ hg --traceback --cwd b commit -Am "merged"
push
$ hg --traceback --cwd b --config notify.fromauthor=True push ../a 2>&1 |
> "$PYTHON" $TESTDIR/unwrap-message-id.py | \
> "$PYTHON" -c 'import sys,re; print(re.sub("\n\t", " ", sys.stdin.read()), end="")'
pushing to ../a
searching for changes
adding changesets
adding manifests
adding file changes
added 4 changesets with 4 changes to 1 files
MIME-Version: 1.0
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: 7bit
Date: * (glob)
Subject: * (glob)
From: committer_1
X-Hg-Notification: changeset 84e487dddc58
Message-Id: <*> (glob)
To: baz, foo@bar
changeset 84e487dddc58 in $TESTTMP/a
details: $TESTTMP/a?cmd=changeset;node=84e487dddc58
summary: newfile
changeset b29c7a2b6b0c in $TESTTMP/a
details: $TESTTMP/a?cmd=changeset;node=b29c7a2b6b0c
summary: x
changeset 0957c7d64886 in $TESTTMP/a
details: $TESTTMP/a?cmd=changeset;node=0957c7d64886
summary: y
changeset 485b4e6b0249 in $TESTTMP/a
details: $TESTTMP/a?cmd=changeset;node=485b4e6b0249
summary: merged
diffs (7 lines):
diff -r ba677d0156c1 -r 485b4e6b0249 b
--- /dev/null Thu Jan 01 00:00:00 1970 +0000
+++ b/b Thu Jan 01 00:00:00 1970 +0000
@@ -0,0 +1,3 @@
+x
+b
+y
$ hg --cwd a rollback
repository tip rolled back to revision 1 (undo push)