peer-or-repo: make sure object in "scheme" have a `instance` object The previous form of having heterogeneous object in the dictionnary makes things more complicated than they needed to be. I am not super happy about the current (especially around 'islocal', that most item do not have), but this is already much better.

peer-or-repo: move the object setup in its own function The `_peerorrepo` function is problematic, because it can build different types of object (repository and peer). This make it hard to adjust the arguments to the type of object we needs. So this patch start a series of change to create peer and repo without going through a common function. We move the part of the function doing object setup it its own function to make it simpler to reuse in others contexts.

tests: use an all too familiar executable in test-run-tests.t (issue6661) true(1) sometimes lives in /usr/bin/, and so this test fails on such systems. We also can't use which(1), because it's apparently not POSIX and Debian complains about it [1]. We also cannot really create a script and chmod +x it, because this is a symlink case, execbit case is slightly below. So let's use something that we know is executable, but not hg itself.

tests: add the missing space to test-hghave.t (issue6762) log() in run-tests.py add a space to the end of all messages that it prints, and I missed this fact in my previous patch.

delta-find: use a single snapshot cache when applying a group to an object This will avoid walking the revlog over and over again in some situations. The difference is hard to show in our current benchmark suite, as the gain is lower than their overall instability.

delta-find: make sure we only use newer full snapshot as candidate The current code does not needs to protect against this, as there are no older snapshot in the current cache. However as we are getting ready to reuse this cache from one revision to another, we need the code to protect itself about what's coming.

delta-find: use sets instead of list in the snapshot cache This seems more appropriate.

delta-find: use a smarter object for snapshot caching This open the way for a longer lived cache.

find-delta: pass the cache-delta usage policy alongside the cache-delta The idea is to give higher level code more control to what will happens with the cache delta passed. This should help with controling how we treat delta's from different sources. The final goal of this change is to allow for server modes where the client can blindly accept any server delta without regards to any local constraints. This will be implemented in later changesets.

find-delta: move most of the debug-find-delta code in the debug module Lets us that module more. It will help us to keep revlog implementation details close to each other.

find-delta: minor preparatory change We are about to add more item in the cachedelta object, so lets access the item by index instead of doing a full extensions.

find-delta: rename _isgooddeltainfo Lets move to a more readable name now that we are allowed to. This cannot hurt.

test-revlog-raw: drop the overwrite of dead code The revlog class no longer have a _isgooddeltainfo method for a long time. So overwriting it does not get us anything. The test have been wrapping the right code since then anyway.

emitrevision: consider ancestors revision to emit as available base This should make more delta base valid. This notably affects: * case where we skipped some parent with empty delta to directly delta against an ancestors * case where an intermediate snapshots is stored. This change means we could sent largish intermediate snapshots over the wire. However this is actually a sub goal here. Sending snapshots over the wire means the client have a high odd of simply storing the pre-computed delta instead of doing a lengthy process that will… end up doing the same intermediate snapshot. In addition the overall size of snapshot (or any level) is "only" some or the overall delta size. (0.17% for my mercurial clone, 20% for my clone of Mozilla try). So Sending them other the wire is unlikely to change large impact on the bandwidth used. If we decide that minimising the bandwidth is an explicit goal, we should introduce new logic to filter-out snapshot as delta. The current code has no notion explicite of snapshot so far, they just tended to fall into the wobbly filtering options. In some cases, this patch can yield large improvement to the bundling time: ### data-env-vars.name = mozilla-try-2019-02-18-zstd-sparse-revlog # benchmark.name = perf-bundle # benchmark.variants.revs = last-100000 before: 68.787066 seconds after: 47.552677 seconds (-30.87%) That translate to large improvement to the pull time : ### data-env-vars.name = mozilla-try-2019-02-18-zstd-sparse-revlog # benchmark.name = pull # benchmark.variants.issue6528 = disabled # benchmark.variants.revs = last-100000 before: 142.186625 seconds after: 75.897745 seconds (-46.62%) No significant negative impact have been observed.

sqlitestore: add an `ancestors` method We will need it during bundling. The implementation mirror the one in revlog.

emitrevision: if we need to compute a delta on the fly, try p1 or p2 first Falling back to `prev` does not yield any real value on modern storage and result in pathological changes to be created on the other side. Doing a delta against a parent will likely be smaller (helping the network) and will be safer to apply on the client (helping future pulls by Triggering intermediate snapshop where they will be needed by later deltas).

emitrevision: simplify the fallback to computed delta Not using the stored delta, or having a full snapshot on disk behave the same ways, so lets use the same code path for that, this is simpler, and it update will be simpler.

emitrevision: also check the parents in the availability closure One of the point of having a closure is to gather the logic in it. So we gather the logic. The `parents[:]` part is a bit ugly but will be replaced by better code soon anyway.

emitrevision: add a small closure to check if a base is usable We will make more use of this and make it more complex too.

reuse-delta-base: improves some documentation The current code got me a bit confused initially. So a bit more documentation around it cannot hurt.

tests: expect the message from 1baf0fffd82f in test-hghave.t (issue6762) Couldn't reproduce locally, but apparently this message may occasionally pop up when running this test.

branching: merge stable into default

changelog-v2: add a configuration to disable rank computation We encountered a graph where rank computation was pathologically slow. We add an option to disable this computation while this is getting fixed. Disabling the rank computation should allow for testing other changelog-v2 features undisturbed (like changeset-based copy tracing). I am purposely not adding a test for the new non-default code path, as this is a temporary work around of an experimental feature.

debugrevlog: display total stored information This is an interesting statistis, so let's display it.

debug-revlog: details about non-ancestors delta-bases Deltas against a base that is not an ancestor of the revision that owns this delta are notable. For example, they introduce complexity during the bundling process as the base might not exist on the unbundling side. We detect them in `hg debugrevlog` and print information about them.

debug-revlog: move the code in revlogutils module We have a module dedicated to debug code, let us use it.

debug-revlog: move the --dump code in `revlogutils` module We have a module dedicated to debug code, let us use it.

delta-find: set the default candidate chunk size to 10 I ran performance and storage tests on repositories of various sizes and shapes for the following values of the config : 5, 10, 20, 50, 100, no-chunking The performance tests do not show any statistical impact on computation times for large pushes and pulls. For searching for an individual delta, this can provide a significant performance improvement with a minor degradation of space-quality on the result. (see data at the end of the commit). For overall store size, the change : - does not have any impact on many small repositories, - has an observable, but very negligible impact on most larger repositories. - One private repository we use for testing sees a small increase in size (1%) in the narrower version. We will try to get more numbers on a larger version of that repository to make sure nothing pathological happens. We pick "10" as the limit as "5" seems a bit more risky. There are room to improve the current code, by using more aggressive filtering and better (i.e any) sorting of the candidates. However this is already a large improvement for pathological cases, with little impact in the common situations. The initial motivation for this change is to fix performance of delta computation for a file where the previous code ended up testing 20 000 possible candidate-bases in one go, which is… slow. This affected about ½ of the file revisions leading to atrocious performance, especially during some push/pull operations. Details about individual delta finding timing: ---------------------------------------------- The vast majority of benchmark cases are unchanged but the three below. The first two do not see any impact on the final delta. The last one sees a change in delta-size that is negligible compared to the full text size. ### data-env-vars.name = mozilla-try-2019-02-18-zstd-sparse-revlog # benchmark.name = perf-delta-find # benchmark.variants.rev = manifest-snapshot-many-tries-a (revision 756096) ∞: 5.844783 5: 4.473523 (-23.46%) 10: 4.970053 (-14.97%) 20: 5.770386 (-1.27%) 50 5.821358 100: 5.834887 MANIFESTLOG: rev = 756096: (no-limit) delta-base = 301840 search-rounds = 6 try-count = 60 delta-type = snapshot snap-depth = 7 delta-size = 179 MANIFESTLOG: rev=756096: (limit = 10) delta-base=301840 search-rounds=9 try-count=51 delta-type=snapshot snap-depth=7 delta-size=179 ### data-env-vars.name = mozilla-try-2019-02-18-zstd-sparse-revlog # benchmark.name = perf-delta-find # benchmark.variants.rev = manifest-snapshot-many-tries-d (revision 754060) ∞: 5.017663 5: 3.655931 (-27.14%) 10: 4.095436 (-18.38%) 20: 4.828949 (-3.76%) 50 4.987574 100: 4.994889 MANIFESTLOG: rev=754060: (no limit) delta-base=301840 search-rounds=5 try-count=53 delta-type=snapshot snap-depth=7 delta-size = 179 MANIFESTLOG: rev=754060: (limite = 10) delta-base=301840 search-rounds=8 try-count=45 delta-type=snapshot snap-depth=7 delta-size = 179 ### data-env-vars.name = mozilla-try-2019-02-18-zstd-sparse-revlog # benchmark.name = perf-delta-find # bin-env-vars.hg.flavor = rust # benchmark.variants.rev = manifest-snapshot-many-tries-e (revision 693368) ∞: 4.869282 5: 2.039732 (-58.11%) 10: 2.413537 (-50.43%) 20: 4.449639 (-8.62%) 50 4.865863 100: 4.882649 MANIFESTLOG: rev=693368: delta-base=693336 search-rounds=6 try-count=53 delta-type=snapshot snap-depth=6 full-test-size=131065 delta-size=199 MANIFESTLOG: rev=693368: delta-base=278023 search-rounds=5 try-count=21 delta-type=snapshot snap-depth=4 full-test-size=131065 delta-size=278 Raw data for store size (in bytes) for various chunk size value below: ---------------------------------------------------------------------- 440 134 384 5 pypy/.hg/store/ 440 134 384 10 pypy/.hg/store/ 440 134 384 20 pypy/.hg/store/ 440 134 384 50 pypy/.hg/store/ 440 134 384 100 pypy/.hg/store/ 440 134 384 ... pypy/.hg/store/ 666 987 471 5 netbsd-xsrc-2022-11-15/.hg/store/ 666 987 471 10 netbsd-xsrc-2022-11-15/.hg/store/ 666 987 471 20 netbsd-xsrc-2022-11-15/.hg/store/ 666 987 471 50 netbsd-xsrc-2022-11-15/.hg/store/ 666 987 471 100 netbsd-xsrc-2022-11-15/.hg/store/ 666 987 471 ... netbsd-xsrc-2022-11-15/.hg/store/ 852 844 884 5 netbsd-pkgsrc-2022-11-15/.hg/store/ 852 844 884 10 netbsd-pkgsrc-2022-11-15/.hg/store/ 852 844 884 20 netbsd-pkgsrc-2022-11-15/.hg/store/ 852 844 884 50 netbsd-pkgsrc-2022-11-15/.hg/store/ 852 844 884 100 netbsd-pkgsrc-2022-11-15/.hg/store/ 852 844 884 ... netbsd-pkgsrc-2022-11-15/.hg/store/ 1 504 227 981 5 netbeans-2018-08-01-sparse-zstd/.hg/store/ 1 504 227 871 10 netbeans-2018-08-01-sparse-zstd/.hg/store/ 1 504 227 813 20 netbeans-2018-08-01-sparse-zstd/.hg/store/ 1 504 227 813 50 netbeans-2018-08-01-sparse-zstd/.hg/store/ 1 504 227 813 100 netbeans-2018-08-01-sparse-zstd/.hg/store/ 1 504 227 813 ... netbeans-2018-08-01-sparse-zstd/.hg/store/ 3 875 801 068 5 netbsd-src-2022-11-15/.hg/store/ 3 875 696 767 10 netbsd-src-2022-11-15/.hg/store/ 3 875 696 757 20 netbsd-src-2022-11-15/.hg/store/ 3 875 696 653 50 netbsd-src-2022-11-15/.hg/store/ 3 875 696 653 100 netbsd-src-2022-11-15/.hg/store/ 3 875 696 653 ... netbsd-src-2022-11-15/.hg/store/ 4 531 441 314 5 mozilla-central/.hg/store/ 4 531 435 157 10 mozilla-central/.hg/store/ 4 531 432 045 20 mozilla-central/.hg/store/ 4 531 429 119 50 mozilla-central/.hg/store/ 4 531 429 119 100 mozilla-central/.hg/store/ 4 531 429 119 ... mozilla-central/.hg/store/ 4 875 861 390 5 mozilla-unified/.hg/store/ 4 875 855 155 10 mozilla-unified/.hg/store/ 4 875 852 027 20 mozilla-unified/.hg/store/ 4 875 848 851 50 mozilla-unified/.hg/store/ 4 875 848 851 100 mozilla-unified/.hg/store/ 4 875 848 851 ... mozilla-unified/.hg/store/ 11 498 764 601 5 mozilla-try/.hg/store/ 11 497 968 858 10 mozilla-try/.hg/store/ 11 497 958 730 20 mozilla-try/.hg/store/ 11 497 927 156 50 mozilla-try/.hg/store/ 11 497 925 963 100 mozilla-try/.hg/store/ 11 497 923 428 ... mozilla-try/.hg/store/ 10 047 914 031 5 private-repo 9 969 132 101 10 private-repo 9 944 745 015 20 private-repo 9 939 756 703 50 private-repo 9 939 833 016 100 private-repo 9 939 822 035 ... private-repo

delta-find: add a way to control the number of bases tested at the same time See inline comment for details. The feature is currently disabled, but should be enabled by default to mitigate some existing pathological cases. Also see the next changeset for details.

test: adjust test-push-race.t timeout's to overall test timeout The generic `tests/testlib/wait-on-file` mechanism scale its timeout with the value of `HGTEST_TIMEOUT`, the `delaypush.py` in `test-push-race.t` is not doing this, and we have been seeing more and more timeout from loaded CI worker lately. Adding this timeout scaling should help with that.

setup: include vendored 3rd party type stubs While pytype may not support PEP 561, PyCharm does, so having the stubs available means it can determine `foo = attr.ib(type=int)` means `foo` is an int. This only applies when using Mercurial as a library, like with TortoiseHg development- PyCharm is already smart enough to use the *.pyi files in the Mercurial source tree when hacking on Mercurial itself. I left the mercurial.cext stubs out because it seems very low level, that 3rd parties shouldn't be using directly.

ci: bump pytype to 2022.11.18 No particular reason, other than the current build is fairly old. It flagged a few more things (that weren't errors based on the logic around them), but OTOH, some of the pyi stubs it generates are less specific.

ci: run the script to add vendored type stubs to typeshed Since CI runs from docker images, it doesn't matter that this mucks with the typeshed bundled with pytype.

contrib: add a script for adding vendored type stubs to typeshed I really hate this, but pytype doesn't support PEP 561 and doesn't seem to have the equivalent of `MYPYPATH` to point to custom stubs. Ignoring the vendored stubs isn't necessarily harmful, but pytype has been choking on the vendored attr package after pytype 2022.03.29 with errors like this: File "/mnt/c/Users/Matt/hg/mercurial/linelog.py", line 52, in __iter__: Built-in function iter was called with the wrong arguments [wrong-arg-types] Expected: (collection: bytearray) Actually passed: (collection: mercurial.thirdparty.attr._make._CountingAttr) File "/mnt/c/Users/Matt/hg/mercurial/dirstateutils/v2.py", line 143, in pack: Built-in function len was called with the wrong arguments [wrong-arg-types] Expected: (obj: Sized) Actually passed: (obj: mercurial.thirdparty.attr._make._CountingAttr) Attributes of protocol Sized are not implemented on mercurial.thirdparty.attr._make._CountingAttr: __len__ File "/mnt/c/Users/Matt/hg/mercurial/dirstateutils/v2.py", line 144, in pack: No attribute 'rfind' on mercurial.thirdparty.attr._make._CountingAttr [attribute-error] File "/mnt/c/Users/Matt/hg/mercurial/dirstateutils/v2.py", line 146, in pack: Built-in function len was called with the wrong arguments [wrong-arg-types] Expected: (obj: Sized) Actually passed: (obj: mercurial.thirdparty.attr._make._CountingAttr) Attributes of protocol Sized are not implemented on mercurial.thirdparty.attr._make._CountingAttr: __len__ File "/mnt/c/Users/Matt/hg/mercurial/dirstateutils/v2.py", line 152, in pack: No attribute 'v2_data' on mercurial.thirdparty.attr._make._CountingAttr [attribute-error] File "/mnt/c/Users/Matt/hg/mercurial/util.py", line 2817, in go: unsupported operand type(s) for /: 'count: mercurial.thirdparty.attr._make._CountingAttr' and 'float: float' [unsupported-operands] No attribute '__truediv__' on 'count: mercurial.thirdparty.attr._make._CountingAttr' or '__rtruediv__' on 'float: float' Called from (traceback): line 2981, in __bytes__ This is essentially the same hack we've been using in TortoiseHg to add the vendored PyQt5 stubs. What I don't understand is pytype *still* generates *.pyi files under .pytype/pyi/mercurial/thirdparty/attr, even when the package is explicitly ignored in the pytype command line args. But it avoids the errors, which means we aren't stuck on pytype==2022.03.29. https://github.com/google/pytype/issues/151

contrib: update check-pytype.sh to list stubs that caused pytype to crash The same logic is in the TortoiseHg tests for running pytype, and it's useful to know if a new version of pytype is better or worse.

typing: add py.typed to mercurial.cext for PEP 561 support Unfortunately, pytype doesn't support this yet. But it was included with the attr package, so we might as well do it here for consistency. Unlike the attr package, these type hints are only partial, so they are marked as such[1] (but who knows if it matters, given these are C extensions, so no local source code to scan). [1] https://peps.python.org/pep-0561/#partial-stub-packages

typing: add missing signature for mercurial.cext.parsers.parse_index2() Flagged by pytype 2022.11.18 when the cext stubs are made visible to it. There are very likely other signatures that are missing, but this is enough to keep it happy for now.

typing: minor tweaks to allow updating to pytype 2022.11.18

python-compat: adapt to Python 3.11 BC breakage with `random.sample` As per https://docs.python.org/3/whatsnew/3.11.html#porting-to-python-3-11: "The population parameter of `random.sample()` must be a sequence, and automatic conversion of sets to lists is no longer supported. Also, if the sample size is larger than the population size, a `ValueError` is raised"

typing: add type hints to mercurial/help.py Was hoping to find more issues like f09bc2ed9100, but it may be that nothing checks the args to that operation. In any event, the work is done and pytype doesn't do a very good job inferring the types. A few of th emore complicated things like the command table are left untyped, because they come from modules that aren't typed yet.

match: make the FLAG_RE pattern a raw string PyCharm was complaining about invalid escape sequences since this was added recently in 3eda36e9b3d6.

configitems: add a default value for "merge-tools.xxx.regappend" When trying to figure out how `hg help -v` took the Set interpolation path in f09bc2ed9100, I turned on devel warnings and noticed this (unrelated) warning: devel-warn: specifying a mismatched default value for a registered config item: 'merge-tools.beyondcompare4.regappend' '' at: c:\Users\Matt\hg\mercurial\filemerge.py:46 (_toolstr) The previous default value for this config was `None`, but that slightly complicates the code at the only site it is used, referenced above.

attr: vendor 22.1.0 The previous version was 5 years old, and pytype 2022.06.30 started complaining about various uses (e.g. seeing `mercurial.thirdparty.attr._make._CountingAttr` instead of `bytearray`). Hopefully this helps. Additionally, this has official python 3.11 support. The `attrs` package is left out, because it is simply a bunch of *.pyi stubs and `from attr.X import *`, and that's not how they've been used up to this point. We'd probably need to customize those anyway to `from mercurial.thirdparty.attr import *`.

tests: update test-util.py for modern attrs package When updating to 22.1.0, this test started failing: Traceback (most recent call last): File "/tmp/mercurial-ci/tests/test-util.py", line 53, in <module> _start_default = (util.timedcmstats.start.default, 'factory') AttributeError: type object 'timedcmstats' has no attribute 'start' Poking around in `hg debugshell`, the attribute is indeed missing, but looks to be attached to `__attrs_attrs__` in both the currently vendored and the modern version of attrs. The old attrs packages will print the same for both accesses, so fingers crossed... >>> print((util.timedcmstats.start.default, 'factory')) (Factory(factory=<function timedcmstats.<lambda> at 0x000001EFDF0F21F0>, takes_self=False), 'factory') >>> print((util.timedcmstats.__attrs_attrs__.start.default, 'factory')) (Factory(factory=<function timedcmstats.<lambda> at 0x000001EFDF0F21F0>, takes_self=False), 'factory')

help: fix a py3 error interpolating Set into b'%s' I can't reproduce it, but a coworker hit this with `hg help -v` with 6.2.3: ... File "mercurial\help.pyc", line 865, in helplist TypeError: %b requires a bytes-like object, or an object that implements __bytes__, not 'set' I can confirm that the original expression fails in `hg debugshell`, and the new one works. The second instance was found by searching for "%s", but PyCharm detects a lot of variables as Any type, so I have no idea if there are other lurking problems.

rhg: upgrade the remainder of the dependencies These are painless, so they are all grouped in this changeset.

rhg: upgrade `clap` dependency This one is the worst one to upgrade since v2 -> v4 broke a ton of API, which thankfully seems saner now. Contrary to what was done in the `hg-core/src/examples/nodemap` rewrite, we're not switching from the "builder" pattern to the "derive" pattern, since that would imply a much larger diff. It can be done incrementally.

hg-cpython: upgrade dependencies `hg-cpython` has no BC breaking dependencies, we can group them all in this changeset.

hg-core: upgrade all remaining dependencies Finally, these dependencies do not require any code changes, so they are all grouped in the same changeset.

hg-core: upgrade `clap` dependency Upgrading is a finally possible now that we're supporting a more recent version of Rust. This required changes in the `nodemap` example.

hg-core: upgrade `zstd` dependency Now that we support a newer version of Rust, we can update this dependency to get all the latest bugfixes and improvements. A slight API adjustment was needed.

hg-core: make use of `strip_suffix` now that we're using Rust 1.45+

rust: use `matches!` macro now that we're using Rust 1.42+

hg-core: remove unneeded util now that we support Rust 1.42+

hg-core: remove unneeded trait now that we support Rust 1.52+

rust: remove newly redundant `use` statements with the 2021 edition prelude https://doc.rust-lang.org/edition-guide/rust-2021/prelude.html

rust: move all crates in the main workspace to edition 2021 We've changed our minimum Rust version to 1.61.0 in the previous patch, and edition 2021 predates that version.

rust: upgrade supported Rust toolchain version A few months ago¹, a decision was made to move the Rust toolchain target to whatever Debian Testing was tracking. I didn't have the bandwidth to act on it until now. This is starting to be even more problematic than before, now that edition 2021 is out. The CI has been updated to track the current Debian testing version, 1.61.0. [1] https://lists.mercurial-scm.org/pipermail/mercurial-packaging/2022-April/000338.html

branching: merge stable into default

tests: fix test-sparse-revlog This one is not covered by the CIbecause I requires an expensive artifact to be cached. So it goes out of think on regular basis (we should fix that…) The test ouput was affected by e706bb41fdb3 as we filtering now happens sooner, removing for the output.

Added signature for changeset c890d8b8bc59

Added tag 6.3.1 for changeset c890d8b8bc59

relnotes: add 6.3.1

memory-usage: fix `hg log --follow --rev R F` space complexity When running `hg log --follow --rev REVS FILES`, the log code will walk the history of all FILES starting from the file revisions that exists in each REVS. Before doing so, it looks if the files actually exists in the target revisions. To do so, it opens the manifest of each revision in REVS to look up if we find the associated items in FILES. Before this changeset this was done in a way that created a changectx for each target revision, keeping them in memory while we look into each file. If the set of REVS is large, this means keeping the manifest for each entry in REVS in memory. That can be large… if REV is in the form `::X`, this can quickly become huge and saturate the memory. We have seen usage allocating 2GB per second until memory runs out. So this changeset invert the two loop so that only one revision is kept in memory during the operation. This solve the memory explosion issue.

tests: run many tests in $TESTTMP/repo instead of $TESTTMP This is useful so we can store other files in $TESTTMP (in particular tests that use docket files (nodemap, dirstate-v2) keep file uids in $TESTTMP/UID)

tests: fix the detection of dirstate-v2 in hghave.py

dirstate-v2: do not put the dirstate data file in a transaction, since the transaction reverts the store, while the dirstate is stored separately

commit: properly consider file include and exclude options when closing branch It looks like this is meant to prevent adding another commit that does nothing but close a branch on top of a commit that already closed the branch. The matcher building functions want `Dict[bytes, Any]`, not `Dict[str, Any]`, which was found by adding type hints to the matcher related methods in scmutil.

tests: demonstrate a bug blocking a redundant branch close

delta-find: adjust the moment when we mark something as "tested" In a coming change, not all elements of `group` might get tested. So we need to have more control about when a revision is actually added to the `tested` set. So we move to a more verbose (and more fragile) version.

delta-find: rename a variable for clarity the index in the delta-chain is also the snapshot depth. So we rename the variable for clarity.

delta-find: small documentation update This is not a 1-1 mapping, but a 1-n mapping. Lets make the associated comment clearer.

delta-find: move pre-filtering with other pre-filtering logic This is more consistent and will help use to be in a clean state before dealing with the "too large group" issue. As a side effect, the debug output now skip some useless cases, making it more useful.

delta-find: expand a function definition and call before extendin it This make the next changeset more compact.

debug: add an option to display statistic about a unbundling operation This will helps a lot to understand how the bundling decision have actually impacted pull/unbundle on the other side.

debug: add an option to display statistic about a bundling operation This will helps a lot to understand how the bundling decision might impact pull/unbundle on the other side.

delta-find: add debug information about reuse of cached data This will help us to understand the behavior of find-delta during a pull.

tests: stop creating temporary files in TESTDIR

cffi: fix a bytes vs str issue on macOS when listing directories This code hasn't been touched in recent years, and the other implementation return bytes for the filename, so I assume this is a holdover from the py2 days. I was unable to test it on mac though, because the `_osutil` import failed.

packaging: refresh dependency hashes (issue6750) Also, add some documentation to the `.in` files.

matcher: do not prepend '.*' to pattern using ^ after flags Since the previous commit (fixing wider issue), the code generated strange regex. This is now fixed and tested.

matcher: fix the issue with regex inline-flag in rust oo Same problem same solution.

matcher: fix issues regex flag contained in pattern (issue6759) Python 3.11 is now enforcing that flag must be at the beginning of the regex This creates a serious regression for people using Python 3.11 with an hgignore using flag in a "relre" pattern. We now detect any flags in such pattern and "prepend" our ".*" pattern after them. In addition, we now insert the flag in the regexp to only affect the pattern we are rewriting. Otherwise, the regex built from the combined pattern would these flags in the middle of it anyway. As a side effect of this last change, we fix a bug… before this change regex flag in a pattern would affect all combined patterns. That was bad and is not longer the case. The Rust code needs to be updated to fix that very bug, but we will do it in another changeset.

release: removed the 6.3.0 tag The revision having two tags confuses some of the build script trying to fetch the version. So, remove the "bad" tag for now.

cffi: fix a bytes vs str issue on macOS when listing directories This code hasn't been touched in recent years, and the other implementation return bytes for the filename, so I assume this is a holdover from the py2 days. I was unable to test it on mac though, because the `_osutil` import failed.

typing: fix the typehint for `skip` arg on `osutil.listdir()` to be optional The cffi and pure implementations both default to `None`, and the args are parsed in C with `"y#|OO:listdir"`, with the C variable initialized to NULL.

typing: add type hints to mpatch implementations Again, using `merge-pyi` to apply the stubs in cext and then manually type the private methods. The generated stub without these hints inferred very little, and the stuff it did was wrong.

typing: add type hints to bdiff implementations Not super important code, but this was an exercise in using `merge-pyi` to fold type stubs back into the code on something small. The cext stubs don't seem to be getting used (at least the only thing in `.pytype/pyi/mercurial/cext` after a run generating the stubs is `__init__.pyi`), so maybe this will help some.

cffi: adjust the list returned by bdiff.blocks to never have a None entry This was flagged by pytype after merging the corresponding bdiff.pyi in cext: File ".../mercurial/cffi/bdiff.py", line 44, in blocks: bad return type [bad-return-type] Expected: List[Tuple[int, int, int, int]] Actually returned: List[None] AFAICT, all callers immediately unpack the tuple into 4 variables, so a `None` entry would simply crash if they aren't all overwritten. As long a `count` and the link list are consistent, this shouldn't be a problem. This placates both pytype and PyCharm (which complained about the `i` in `rl[i]` having the wrong type with the old code).

typing: fix a syntax error in mercurial/cext/bdiff.pyi I noticed because `merge-pyi` on the non-cext implementations with this file as input skipped the return type for this. `pytype-single --parse-pyi` confirmed it was a problem.

hg-core: relax dependencies pinning Being this rigid makes packagers' job more difficult since they might not carry the exact version. This hard pinning was introduced in eb02decdf but wasn't strictly necessary to achieve its compatibility goal.

tests: make running ls in a no longer existing directory more portable On Linux, ls -A simply returns nothing and the exit code is 0. On NetBSD, ls -A complains that . doesn't exist and the exit code is 1. Sadly, it's not possible to do something like "[1] (?)", so " || true" is the best I could come up with.

tests: move some lines inside #if windows-#else block test-removeemptydirs.t This is done to make it's clear that windows is not affected by this test case, IOW windows and non-windows platforms are now tested separately, because their results are very different.

tests: use ls -A instead of ls -1 in test-removeemptydirs.t In case the tests are run as root, ls assumes -A by default on some systems (e.g. NetBSD). Tests probably shouldn't be run as root, but let's use -A just in case, for convenience.

release: add the "6.3" to the 6.3 release The release was tagged "6.3.0" while the custom have been to omit the third ".0" for initial release for the history of the project. For consistency, we add the customary version of the tag to the 6.3 release.

tests: fix new git protocol policy in convert-git.t Recent versions of git restrict the use of the "file" protocol for security reasons (https://github.com/git/git/commit/a1d4f67). This broke this test, which failed with the following hidden error: "fatal: transport 'file' not allowed". This patch relaxes the git configuration for the test to solve this.

debug-delta-find: add a --source option This will help us to understand the delta-find operation in different situations.

demandimport: ensure lazyloaderex sets loader attributes (issue6725) Adds test capturing missed expectation.

shelve: add test for Shelf.changed_files

shelve: add Shelf.changed_files for resolving changed files in a plugin

branching: merge stable into default

relnotes: add final 6.3 relnotes

Added signature for changeset 04f1dba53c96

Added tag 6.3.0 for changeset 04f1dba53c96

rust: create wrapper struct to reduce `regex` contention issues Explanations inline.

tags-fnode-cache: skip building a changectx in getfnode Building a changectx object is costly, doing it just to retrieve the revision number is suboptimal. Directly fetching the revision number from the changelog provide a sizeable speedup to `hg debugupdatecache`. ### data-env-vars.name = mercurial-2018-08-01-zstd-sparse-revlog # benchmark.name = debug-update-cache # benchmark.variants.pre-state = warm before: 0.213229 seconds after: 0.165577 seconds (-22.35%) # data-env-vars.name = mercurial-filtered-2019-11-22-zstd-sparse-revlog before: 1.200383 seconds after: 1.071618 seconds (-10.73%) # data-env-vars.name = mozilla-central-2018-08-01-zstd-sparse-revlog before: 1.465735 seconds after: 0.923128 seconds (-37.02%) # data-env-vars.name = mozilla-try-2019-02-18-zstd-sparse-revlog before: 6.511771 seconds after: 4.507316 seconds (-30.78%) # data-env-vars.name = netbeans-2018-08-01-zstd-sparse-revlog before: 1.023007 seconds after: 0.645026 seconds (-36.95%) # data-env-vars.name = pypy-2018-08-01-zstd-sparse-revlog before: 0.381141 seconds after: 0.268654 seconds (-29.51%)

tags-fnode-cache: do not repeatedly open the filelog in a loop While getting multiple hgtagsfnodecache entries, we were opening (and closing) the `.hgtags` filelog for each iteration. The meant repeatedly reading and parsing the version same information from disk. A quite costly operation. We no longer do this, leading to a sizable improvement in `hg debugupdatecache` run for an already warm repositories. ### data-env-vars.name = mercurial-2018-08-01-zstd-sparse-revlog # benchmark.name = debug-update-cache # benchmark.variants.pre-state = warm before: 1.711778 seconds after: 0.213229 seconds (-87.54%) # data-env-vars.name = pypy-2018-08-01-zstd-sparse-revlog before: 4.010817 seconds after: 0.381141 seconds (-90.50%) # data-env-vars.name = netbeans-2018-08-01-zstd-sparse-revlog before: 13.574141 after: 1.023007 seconds (-92.46%) # data-env-vars.name = mozilla-central-2018-08-01-zstd-sparse-revlog before: 18.884656 after: 1.465735 seconds (-92.24%) # data-env-vars.name = mozilla-try-2019-02-18-zstd-sparse-revlog before: 88.924823 after: 6.511771 seconds (-92.68%)

profile: prevent a crash when line number is unknown For some unknown reason, `self.lineno` can be None. The previous code crashed in such case, we now ignore the case, as we do for other error in this function. We also fallback to using `-1` in the output when this lack of line number makes it to the display code. The reason of unknown line-numbers is… unknown.

configitems: enable pullbundles by default The use of pullbundles is already protected: they are only used when a pullbundles.manifest file is created on the server. Having an additional flag doesn't really make sense and can confuse users (as indicated in the mercurial mailing list on the topic "Can't get pull-bundles working").

upgrade: byteify requirement changes written to output ui.write() expects bytes, and internally uses `b''` as the default when getting the `label` keyword from `*args`. So either we're missing test coverage, or there's some very subtle conversion from unicode somewhere. Also, slip in a type hint to flag this in the future.

typing: add basic type hints to stringutil.py

vfs: make the default opener mode binary The default was already binary for `abstractvfs`, and the `vfs` implementation adds binary mode if the caller didn't supply it. Therefore, it should be safe for all vfs objects (and I don't think we want text reads anyway).

typing: add basic type hints to vfs.py Again, there's a lot more that could be done, but this sticks to the obviously correct stuff that is related to primitives or `vfs` objects. Hopefully this helps smoke out more path related bytes vs str issues in TortoiseHg. PyCharm seems smart enough to apply hints from annotated superclass functions, but pytype isn't (according to the *.pyi file generated), so those are annotated too. There was some discussion about changing the default path arg from `None` to `b''` in order to avoid the more verbose `Optional` declarations. This would be more in line with `os.path.join()` (which rejects `None`, but ignores empty strings), and still not change the behavior for callers still passing `None` (because the check is `if path` instead of an explicit check for `None`). But I didn't want to hold this up while discussing that, so this documents what _is_.

util: implement `writelines()` on atomictempfile With typehints on the vfs objects, pytype will flag this: FAILED: /mnt/c/Users/Matt/hg/.pytype/pyi/mercurial/patch.pyi /usr/bin/python3.8 -m pytype.single --imports_info /mnt/c/Users/Matt/hg/.pytype/imports/mercurial.patch.imports --module-name mercurial.patch -V 3.7 -o /mnt/c/Users/Matt/hg/.pytype/pyi/mercurial/patch.pyi --analyze-annotated --nofail --quick /mnt/c/Users/Matt/hg/mercurial/patch.py File "/mnt/c/Users/Matt/hg/mercurial/patch.py", line 535, in writerej: No attribute 'writelines' on mercurial.util.atomictempfile [attribute-error] In Union[ mercurial.util.atomictempfile, mercurial.vfs.checkambigatclosing, mercurial.vfs.delayclosedfile, mercurial.windows.fdproxy, mercurial.windows.mixedfilemodewrapper ] It's not a real problem there (atomictempfile is only created by passing different args), but it's reasonable for this to implement the function and behave like a normal file. There are other functions missing that can be added if/when needed.

typing: add basic type hints to localrepo.py There's a lot more that could be done, but this sticks to the obviously correct stuff that is either related to existing imports or primitives. Hopefully this helps smoke out more path related bytes vs str issues in TortoiseHg. I'm avoiding the interfaces for now, because they seem to confuse pytype and/or PyCharm. It might be worth typing the return of `makelocalrepository` to `localrepository`, but that leaks an implementation detail, so that can be revisited later.

check-code: drop the check for whitespace around named parameters This check flags py3 annotations of named parameters, because `black` adds spaces around the assignment in this case. Since the chosen formatter has opinions (and pylint also wants the space in the case of annotations), drop the check so we can use py3 annotations.

rhg: add a config option to fall back immediately This is useful for debugging the behavior of the "default" `hg` in tests without having to manually substitute the fallback path.

rhg: stop shadowing `exit` function This will be useful for the next patch which needs it.

config: add alias from `hg help rhg` to `hg help rust` This will make using `rhg` more user-friendly and features more discoverable.

rhg: add `config.rhg` helptext This will make using `rhg` more user-friendly and features more discoverable.