perf: clear vfs audit_cache before each run
When generating a stream clone, we spend a large amount of time auditing path.
Before this changes, the first run was warming the vfs cache for the other
runs, leading to a large runtime difference and a "faulty" reported timing for
the operation.
We now clear this important cache between run to get a more realistic timing.
Below are some example of median time change when clearing these cases. The
maximum time for a run did not changed significantly.
### data-env-vars.name = mozilla-central-2018-08-01-zstd-sparse-revlog
# benchmark.name = hg.perf.exchange.stream.generate
# bin-env-vars.hg.flavor = default
# bin-env-vars.hg.py-re2-module = default
# benchmark.variants.version = latest
no-clearing: 17.289905
cache-clearing: 21.587965 (+24.86%, +4.30)
## data-env-vars.name = mozilla-central-2024-03-22-zstd-sparse-revlog
no-clearing: 32.670748
cache-clearing: 40.467095 (+23.86%, +7.80)
## data-env-vars.name = mozilla-try-2019-02-18-zstd-sparse-revlog
no-clearing: 37.838858
cache-clearing: 46.072749 (+21.76%, +8.23)
## data-env-vars.name = mozilla-unified-2024-03-22-zstd-sparse-revlog
no-clearing: 32.969395
cache-clearing: 39.646209 (+20.25%, +6.68)
In addition, this significantly reduce the timing difference between the
performance command, from the perf extensions and a `real `hg bundle` call
producing a stream bundle. Some significant differences remain especially on
the "mozilla-try" repositories, but they are now smaller.
Note that some of that difference will actually not be
attributable to the stream generation (like maybe phases or branch map
computation).
Below are some benchmarks done on a currently draft changeset fixing some
unrelated slowness in `hg bundle` (34a78972af409d1ff37c29e60f6ca811ad1a457d)
### data-env-vars.name = mozilla-central-2018-08-01-zstd-sparse-revlog
# bin-env-vars.hg.flavor = default
# bin-env-vars.hg.py-re2-module = default
hg.perf.exchange.stream.generate: 21.587965
hg.command.bundle: 24.301799 (+12.57%, +2.71)
## data-env-vars.name = mozilla-central-2024-03-22-zstd-sparse-revlog
hg.perf.exchange.stream.generate: 40.467095
hg.command.bundle: 44.831317 (+10.78%, +4.36)
## data-env-vars.name = mozilla-unified-2024-03-22-zstd-sparse-revlog
hg.perf.exchange.stream.generate: 39.646209
hg.command.bundle: 45.395258 (+14.50%, +5.75)
## data-env-vars.name = mozilla-try-2019-02-18-zstd-sparse-revlog
hg.perf.exchange.stream.generate: 46.072749
hg.command.bundle: 55.882608 (+21.29%, +9.81)
## data-env-vars.name = mozilla-try-2023-03-22-zlib-general-delta
hg.perf.exchange.stream.generate: 334.716708
hg.command.bundle: 377.856767 (+12.89%, +43.14)
## data-env-vars.name = mozilla-try-2023-03-22-zstd-sparse-revlog
hg.perf.exchange.stream.generate: 302.972301
hg.command.bundle: 326.098755 (+7.63%, +23.13)
= Mercurial 6.6.3 =
* tests: don't use "status" operand of dd in test-censor.t (issue6858)
* tests: make sha256line.py available for all tests
* tests: use sha256line.py instead of /dev/random in test-censor.t (issue6858)
* tests: fix nondeterministic test failure in test-contrib-perf.t
* rust-changelog: don't panic on empty file lists
* grep: restore usage of --include/--exclude options
= Mercurial 6.6.2 =
* histedit: remove superfluous echo() and endwin() calls (issue6859)
* persistent-nodemap: avoid writing nodemap for empty revlog
* persistent-nodemap: respect the mmap setting when refreshing data
* bundle: do not detect --base argument that match nothing as lack of argument
* narrow: prevent removal of ACL-defined excludes
* pycompat: fix bytestr(bytes) in Python 3.11
= Mercurial 6.6.1 =
The first two patches fix aborted transactions that could happen since 6.6.
* revlog: avoid exposing delayed index entry too widely in non-inline revlog
* revlog: avoid wrongly updating the data file location on "divert"
* tests: do not fail tests in a state with uncommitted .py file removal
* perf-tags: fix the --clear-fnode-cache-rev code
* perf-tags: fix clear_cache_fnodes to actually clear that cache
* censor: fix things around inlining
* Various Python 3 cleanups
* Various Windows test suite fixes
= Mercurial 6.6 =
As usual, a *lot* of patches don't make it to this list.
== New Features ==
* rust-config: add support for default config items
* rhg: add config defaults to configitems.toml
* rhg: support "status FILE"
* rhg: support rhg files [FILE]
* commands: add admin namespace
* admin-command: add verify command
* debug-delta-chain: add options to control what we compute
* Added some caching for uncompressed revlog chunks that speed up some operations
== New Experimental Features ==
== Bug Fixes ==
* Improve OpenVMS support
* byteify-strings: passe sysstr to attr function and wrapper
* journal: track bookmark deletion
* transaction: actually delete file created during the transaction on rollback
* pull: avoid referencing a variable before it is initialized
* rust-filepatterns: also normalize RelPath
* narrow: hoist a variable to a higher level to avoid use-before-init warning
* convert: stabilize subversion date sorter
* run-tests: detect HGWITHRUSTEXT value
* censor: accept censored revision during upgrade
* rust-matchers: fix quadratic complexity in `FileMatcher`
* unstable: do not consider internal phases when computing unstable
* Still a whole bunch of Python 2 -> 3 migration leftovers
== Backwards Compatibility Changes ==
* perf: display all timing by default
* infinitepush: drop the extension
* debug-delta-chain: print less data by default
== Internal API Changes ==
* The revlog interface has been significantly cleaned up.
* configitems: add `documentation` field
* configitems: declare items in a TOML file
* configitems: move blackbox's config items to the new configitems.toml
* path-suboption: deprecated specifying the attributes as bytes
* pycompat: deprecate using bytes
* check-code: drop the `safehasattr` rule
* safehasattr: drop usage in favor of hasattr
* rust: add `UncheckedRevision` type
* rust: implement the `Graph` trait for all revlogs
* rust: make `Revision` a newtype
* perf: change the way we approach revlog reading
* perf: ensure all readlog's reading is done within a `reading` context
* delta-chain: move the debugdeltachain command in revlogutils
== Miscellaneous ==