relnotes/6.5
author Pierre-Yves David <pierre-yves.david@octobus.net>
Tue, 09 Apr 2024 02:54:19 +0200
changeset 51586 1cef1412af3e
parent 51115 c083d9776cb2
permissions -rw-r--r--
phases: rework the logic of _pushdiscoveryphase to bound complexity This rework the various graph traversal in _pushdiscoveryphase to keep the complexity in check. This is done though a couple of things: - first, limiting the space we have to explore, for example, if we are not in publishing push, we don't need to consider remote draft roots that are also draft locally, as there is nothing to be moved there. - avoid unbounded descendant computation, and use the faster "rev between" computation. This provide a massive boost to performance when exchanging with repository with a massive amount of draft, like mozilla-try: ### data-env-vars.name = mozilla-try-2023-03-22-zstd-sparse-revlog # benchmark.name = hg.command.push # bin-env-vars.hg.flavor = default # bin-env-vars.hg.py-re2-module = default # benchmark.variants.explicit-rev = all-out-heads # benchmark.variants.issue6528 = disabled # benchmark.variants.protocol = ssh # benchmark.variants.reuse-external-delta-parent = default ## benchmark.variants.revs = any-1-extra-rev before: 20.346590 seconds after: 11.232059 seconds (-38.15%, -7.48 seconds) ## benchmark.variants.revs = any-100-extra-rev before: 24.752051 seconds after: 15.367412 seconds (-37.91%, -9.38 seconds) After this changes, the push operation is still quite too slow. Some of this can be attributed to general phases slowness (reading all the roots from disk for example) and other know slowness (not using persistent-nodemap, branchmap, tags, etc. We are also working on them, but with this series, phase discovery during push no longer showing up in profile and this is a pretty nice and bit low-hanging fruit out of the way. ### (same case as the above) # benchmark.variants.revs = any-1-extra-rev pre-%ln-change: 44.235070 this-changeset: 11.232059 seconds (-74.61%, -33.00 seconds) # benchmark.variants.revs = any-100-extra-rev pre-%ln-change: 49.234697 this-changeset: 15.367412 seconds (-68.79%, -33.87 seconds) Note that with this change, the `hg push` performance is now much closer to the `hg pull` performance, even it still lagging behind a bit. (and the overall performance are still too slow). ### data-env-vars.name = mozilla-try-2023-03-22-ds2-pnm # benchmark.variants.explicit-rev = all-out-heads # benchmark.variants.issue6528 = disabled # benchmark.variants.protocol = ssh # benchmark.variants.pulled-delta-reuse-policy = default # bin-env-vars.hg.flavor = rust ## benchmark.variants.revs = any-1-extra-rev hg.command.pull: 6.517450 hg.command.push: 11.219888 ## benchmark.variants.revs = any-100-extra-rev hg.command.pull: 10.160991 hg.command.push: 14.251107 ### data-env-vars.name = mozilla-try-2023-03-22-zstd-sparse-revlog # bin-env-vars.hg.py-re2-module = default # benchmark.variants.explicit-rev = all-out-heads # benchmark.variants.issue6528 = disabled # benchmark.variants.protocol = ssh # benchmark.variants.pulled-delta-reuse-policy = default ## bin-env-vars.hg.flavor = default ## benchmark.variants.revs = any-1-extra-rev hg.command.pull: 8.577772 hg.command.push: 11.232059 ## bin-env-vars.hg.flavor = default ## benchmark.variants.revs = any-100-extra-rev hg.command.pull: 13.152976 hg.command.push: 15.367412 ## bin-env-vars.hg.flavor = rust ## benchmark.variants.revs = any-1-extra-rev hg.command.pull: 8.731982 hg.command.push: 11.178751 ## bin-env-vars.hg.flavor = rust ## benchmark.variants.revs = any-100-extra-rev hg.command.pull: 13.184236 hg.command.push: 15.620843

= Mercurial 6.5.3 =

 * "hgweb: pass strings in WSGI environment correctly from wsgicgi"
 * "perf: introduce more cache invalidation option in perf::tags"
 * "perf: add a `--clear-fnode-cache-rev` argument to perf::tags"
 * "perf: add a --update-last flag to perf::tags"
 * "blackbox: add a option to duplicate output to stderr too"
 * "tags: avoid expensive access to repo.changelog in a loop"
 * "revlog: fix a bug where NULL_NODE failed to be resolved to NULL_REV"
 * "httppeer: fix static-http: scheme autodetection (issue6833)"
 * "tests: backed out changeset 8037ddacad47"
 * "stream-clone: fix a crash when a repo with an empty revlog is cloned"
 * "censor: accept censored revision during upgrade"
 * "revlog: avoid opening and closing the file for each cloned revision"

= Mercurial 6.5.2 =

 * hgweb: encode WSGI environment using the ISO-8859-1 codec
 * rhg: fix the bug where sparse config is interpreted as relglob instead of glob
 * gpg: fix an UnboundLocalError whenever using --force
 * transaction: fix __repr__() and make the default name bytes
 * setup: make the error "Unable to find a working hg binary" more informative
 * tests: avoid test environment affecting setup.py
 * run-tests: detect HGWITHRUSTEXT value

= Mercurial 6.5.1 =

 * A bunch of improvements to Python 3.12 compatibility
 * repoview: fix the filter created by `extrafilter`
 * Improve portability of the test suite
 * fncache: fix a bug that corrupts the fncache after transaction rollback
 * revlog: fix the naming scheme use by split temporary file
 * perf: fix perf::tags

= Mercurial 6.5 =

As usual, a lot of patches don't make it to this list since they're more internal.

== New Features ==

 * Improved Python 3.12 compatiblity
 * configitems: enable changegroup3 by default (unless using infinitepush)
 * extras: expose 'retained_extras' for extensions to extend
 * stabletailgraph: implement stable-tail sort
 * stabletailgraph: naive version of leap computation
 * bundle: introduce a "v3" spec
 * clone-bundles: add a basic first version of automatic bundle generation
 * clone-bundles: garbage collect older bundle when generating new ones
 * clone-bundles: only regenerate the clone bundle when cached ration is low
 * clone-bundles: also control automation based on absolute number of revisions
 * clone-bundles: add a configuration to control auto-generation on changes
 * clone-bundles: introduce a command to refresh bundle
 * clone-bundles: add a command to clear all bundles
 * clone-bundles: add an option to generate bundles in the background
 * clonebundles: add support for inline (streaming) clonebundles
 * clonebundles: adds a auto-generate.serve-inline option
 * match: add `filepath:` pattern to match an exact filepath relative to the root
 * hgweb: add "children" into the JSON template for a changeset
 * hgweb: add support to explicitly access hidden changesets
 * pull: add --remote-hidden option and pass it through peer creation
 * hidden: add support for --remote-hidden to HTTP peer
 * hidden: support passing --hidden with `serve --stdio`
 * hidden: add support to explicitly access hidden changesets with SSH peers
 * perf: introduce a `perf::stream-locked-section` command
 * perf: add a function to find a stream version generator
 * perf: add support for stream-v3 during benchmark
 * perf: add a perf::stream-generate command
 * perf: add a perf::stream-consume
 * cli: make debugnodemap capable of inspecting an arbitrary nodemap
 * rust: configure MSRV in Clippy
 * rhg: make `rhg files` work if `ui.relative-files=true` is specified
 * rhg: support `rhg files` with `ui.relative-paths=false`
 * rhg: support `status --print0`
 * tree-manifest: allow `debugupgraderepo` to run on tree manifest repo
 * library: enable runpy invocation on mercurial package
 * library: incorporate demandimport into runpy invocation
 * exchange: allow passing no includes/excludes to `pull()`

== New Experimental Features ==

 * stream-clone: add an experimental v3 version of the protocol
 * stream-clone: support streamv3 on the cli [hg bundle]

== Bug Fixes ==

 * mail: add a missing argument to properly override starttls
 * bundle: include required phases when saving a bundle (issue6794)
 * outgoing: fix common-heads computation from `missingroots` argument
 * strip: do not include internal changeset in the strip backup
 * bundle: abort if the user request bundling of internal changesets
 * bundle: prevent implicit bundling of internal changeset
 * encoding: avoid quadratic time complexity when json-encoding non-UTF8 strings
 * sha1dc: Make sure SHA1DC_BIGENDIAN is set on Darwin/PowerPC
 * zstd: hack include order to ensure that our zstd.h is found
 * dirstate: better error messages when dirstate is corrupted
 * stream-clone: avoid opening a revlog in case we do not need it
 * treemanifest: make `updatecaches` update the nodemaps for all directories
 * rust-hg-core: move from `ouroboros` to `self_cell`
 * rust-dependencies: switch from `users` to `whoami`
 * dirstate-v2: actually fix the dirstate-v2 upgrade race
 * dirstate: avoid leaking disk space in `hg debugrebuilddirstate`
 * clonebundles: add warning if auto-generate is enabled without formats
 * win32mbcs: unbyteify some strings for py3 support
 * rust-revlog: fix incorrect results with NULL_NODE prefixes
 * rust-revlog: fix RevlogEntry.data() for NULL_REVISION

== Backwards Compatibility Changes ==

 * infinitepush: aggressively deprecated infinite push
 * narrow: indicated the default of 'Yes' when confirming auto-remove-includes

== Internal API Changes ==

 * Store walk was reworked to fix small race conditions in stream-clone and
   greatly improve its API robustness and flexibility.

== Miscellaneous ==

 * Typechecking support was improved in a lot of places
 * Removed more useless compat code for now unsupported Python versions
 * Sped up zstd usage in Rust contexts
 * revlog: add an exception hint when processing LFS flags without the extension
 * ui: keep the progress bar around when writing if stdout is not a tty
 * transaction: use a ".bck" extension for all backup file