hgext: start building a library for simple hooks Many workflows depend on hooks to enforce certain policies, e.g. to prevent forced pushes. The Mercurial Guide includes some cases and Google can help finding others, but it can save users a lot of time if hg itself has a couple of examples for further customization. Differential Revision: https://phab.mercurial-scm.org/D6825

exchange: turn on option that makes concurrent pushes work better The motivation is simply to make hg work better out of the box. This is a slight backwards compatibility break, because client extensions could have assumed that the list of heads the client sees during discovery will be the list of heads during the entirety of the push. It seems unlikely to matter, and not worth mentioning. There's a fair amount of diff in tests, but this is just due to sending a few more bytes on the wire, except for test-acl.t. The extra "invalid branch cache" lines in test-acl.t don't seem to indicate a problem: the branchcache now get computed during the bundle application (because of the check:updated-heads bundle part), but doesn't get rolled back when transactions rollback, thus causing a message in the next operation computing the branch cache. Before this change, I assume the branchcache was only computed on transaction commit, so not computed at all when the transactions roll back, thus no messages. Differential Revision: https://phab.mercurial-scm.org/D8202

update: simplify slightly Differential Revision: https://phab.mercurial-scm.org/D8204

help: clarify behavior of server.concurrent-push-mode So it doesn't seemingly say that old clients cannot talk to server configured with concurrent-push-mode=check-related. They can, they just don't get the benefit. Differential Revision: https://phab.mercurial-scm.org/D8201

commit: error out on unresolved files even if commit would be empty Differential Revision: https://phab.mercurial-scm.org/D8195

tests: add test of committing with conflicts but no changes in wdir I'm about to change the behavior slightly here, so let's have a test that shows that. Differential Revision: https://phab.mercurial-scm.org/D8194

transaction: clarify the logic around pre-finalize/post-finalize I am taking a bit more verbose route, but I find it easier to follow for people who (re)discover the code. (This is a gratuitous cleanup I did while looking at something else.) Differential Revision: https://phab.mercurial-scm.org/D8176

transaction: move constant to upper case These constant are internal to the module and can be safely renamed. Having them upper case help to clarify their "constant" status. (This is a gratuitous cleanup I did while looking at something else.) Differential Revision: https://phab.mercurial-scm.org/D8175

tests: handle In-Reply-To headers for line wrapping Python 3 tends to insert a newline for both Message-ID and In-Reply-To headers, so unwrap both. Just check the wrapped line format explicitly without regular expression. Differential Revision: https://phab.mercurial-scm.org/D8171

remotefilelog: add 'changelog' arg to shallowcg1packer.generate (issue6269) This cause traceback on widening using narrow extension when remotefilelog is enabled. Differential Revision: https://phab.mercurial-scm.org/D8134

remotefilelog-test: glob some flaky output line This is similar to ee0959e7d435. The affected line is flaky underload, yet the final result is correct. The command involves background pre-check of output, these are not stable probably because they run in parallel in multiple process. If it become useful to start testing precise internal details of the, they will have to be tested in a more appropriate framework than `.t` tests. Differential Revision: https://phab.mercurial-scm.org/D8179

histedit: py3 fixes for curses mode Differential Revision: https://phab.mercurial-scm.org/D8150

branch: make --force work even when specifying revs The `hg branch` command accepts a `--force` parameter that allows to "set branch name even if it shadows an existing branch". However, before this patch, that didn’t work when specifying revs with `-r`.

tests: use new, use-case-specific methods from merge module Differential Revision: https://phab.mercurial-scm.org/D8169

merge: introduce a merge() for that use-case In the same vein as some earlier patches like f546d2170b0f (merge: introduce a clean_update() for that use-case, 2020-01-15). Differential Revision: https://phab.mercurial-scm.org/D8168

merge: drop redundant mergeforce argument from hg.merge() The only caller that passed a value for either `force` or `mergeforce` passed the same value for both, so let's simplify the interface by accepting only `force`. Differential Revision: https://phab.mercurial-scm.org/D8167

merge: change default of hg.merge()'s "force" argument from None to False The argument is only passed to `mergemode.update()`, and that function treats `None` just like `False`, so `False` seems clearer. Differential Revision: https://phab.mercurial-scm.org/D8166

debugmergestate: make templated Our IntelliJ team wants to be able to read the merge state in order to help the user resolve merge conflicts. They had so far been reading file contents from p1() and p2() and their merge base. That is not ideal for several reasons (merge base is not necessarily the "graft base", renames are not handled, commands like `hg update -m` is not handled). It will get especially bad as of my D7827. This patch makes the output s a templated. I haven't bothered to make it complete (e.g. merge driver states are not handled), but it's probably good enough as a start. I've done a web search for "debugmergestate" and I can't find any indication that any tools currently rely on its output. If it turns out that we get bug reports for it once this is released, I won't object to backing this patch out on the stable branch (and then perhaps replace it by a separate command, or put it behind a new flag). The changes in test-backout.t are interesting, in particular this: ``` - other path: foo (node not stored in v1 format) + other path: (node foo) ``` I wonder if that means that we actually read v1 format incorrectly. That seems to be an old format that was switched away from in 2014, so it doesn't matter now anyway. Differential Revision: https://phab.mercurial-scm.org/D8120

tests: add tests of debugmergestate with unresolved/resolved path conflicts I'm about to change `hg debugmergestate` and it broke on these "pu" and "pr" records on my first attempt (D8113), so let's add test coverage. Differential Revision: https://phab.mercurial-scm.org/D8119

mergestate: determine if active without looking for state files on disk I couldn't think of a reason that we need to check state files on disk to determine if a merge is active. I could imagine them being for there for detecting broken state files that would then be cleaned up by some later command, but we always delete the entire `.hg/merge/` tree, so that doesn't seem to be it. The checks were added in 4e932dc5c113 (resolve: abort when not applicable (BC), 2014-04-18). Perhaps there were needed for that and then made obsolete by 6062593d8b06 (resolve: don't abort resolve -l even when no merge is in progress, 2014-05-23). The reason I want to delete the checks is that I think `ms = mergestate.read(repo); ms.active() and ms.local` should be a valid pattern, but it crashes when the merge state file is an empty file if we consider mere presence of the file as "active". Differential Revision: https://phab.mercurial-scm.org/D8118

phabricator: update the protocol documentation The `branch` property wasn't added to the `hg:meta` example when it was added to the metadata in d49ab47be8a6. Additionally, `properties` in the Differential Revision dict is a dinctionary, not a list. While here, also alphabetize the responses from Phabricator because that's how it is being printed with `hg debugcallconduit`, and this makes it easier to compare. Differential Revision: https://phab.mercurial-scm.org/D8170

relnotes: move entry to the right spot It appears a conflict resolution went wrong. Differential Revision: https://phab.mercurial-scm.org/D8151

revlog-compression: release note entry for update the config to be a list I updated the changeset, but forgot to phabsend apparently. Differential Revision: https://phab.mercurial-scm.org/D8165

rust-nodemap: a method for full invalidation This will be used for exceptional operations, such as a `__delitem__` on the `MixedIndex` with Rust nodemap. In principle, `NodeTree` should also be able to forget an entry in an efficient way, by accepting to insert `Element::None` instead of only `Element::Rev(r)`, but that seems really overkill at this point. We need to support exceptional operations such as `__delitem__`, only for completeness of the revlog index as seen from Python. The Python callers don't seem to even really need it, deciding to drop the nodemap unconditionally at at higher level when calling `hg strip`. Also, `hg strip` is very costly for reasons that are unrelated to nodemap aspects. Differential Revision: https://phab.mercurial-scm.org/D8098

rust-nodemap: accounting for dead blocks By the very append-only nature of the `NodeTree`, inserting new blocks has the effect of making some of the older ones useless as they become unreachable. Therefore some automatic housekeeping will need to be provided. This is standard procedure in the word of databases, under names such as "repack" or "vacuum". The new `masked_readonly_blocks()` will provide callers with useful information to decide if the nodetree is ripe for repacking, but all the `NodeTree` can provide is how many blocks have been masked in the currently mutable part. Analysing the readonly part would be way too long to do it for each transaction and defeat the whole purpose of nodemap persistence. Serializing callers (from the Python layer) will get this figure before each extraction and maintain an aggregate counter of unreachable blocks separately. Note: at this point, the most efficient repacking is just to restart afresh with a full rescan. Differential Revision: https://phab.mercurial-scm.org/D8097

rust-nodemap: core implementation for shortest In this implementation, we just make `lookup()` return also the number of steps that have been needed to come to a conclusion from the nodetree data, and `validate_candidate()` takes care of the special cases related to `NULL_NODE`. This way of doing minimizes code duplication, but it means that the comparatively slower finding of first non zero nybble will run for all calls to `find()` where it is not needed. Still running on the file generated for the mozilla-central repository, it seems indeed that we now get more ofter 320 ns than 310. The odds that this could have a significant impact on real life Mercurial performance are still looking low. Let's wait for actual benchmark runs to see if an optimization is needed here. Differential Revision: https://phab.mercurial-scm.org/D7819

rust-nodemap: special case for prefixes of NULL_NODE We have to behave as though NULL_NODE was stored in the node tree, although we don't store it. Differential Revision: https://phab.mercurial-scm.org/D7798

rust-nodemap: pure Rust example To run, use `cargo run --release --example nodemap` This demonstrates that simple scenarios entirely written in Rust can content themselves with `NodeTree<T>`. The example mmaps both the nodemap file and the changelog index. We had of course to include an implementation of `RevlogIndex` directly, which isn't much at this stage. It felt a bit prematurate to include it in the lib. Here are some first performance measurements, obtained with this example, on a clone of mozilla-central with 440000 changesets: (create) Nodemap constructed in RAM in 153.638305ms (query CAE63161B68962) found in 22.362us: Ok(Some(269489)) (bench) Did 3 queries in 36.418µs (mean 12.139µs) (bench) Did 50 queries in 184.318µs (mean 3.686µs) (bench) Did 100000 queries in 31.053461ms (mean 310ns) To be fair, even between bench runs, results tend to depend whether the file is still in kernel caches, and it's not so easy to get back to a real cold start. The worst we've seen was in the 50us ballpark. In any busy server setting, the pages would always be in RAM. We hope it's good enough not to be significantly slower on any concrete Mercurial operation than the C nodetree when fully in RAM, and of course this implementation has the serious headstart advantage of persistence. Differential Revision: https://phab.mercurial-scm.org/D7797

rust-nodemap: input/output primitives These allow to initiate a `NodeTree` from an immutable opaque sequence of bytes, which could be passed over from Python (extracted from a `PyBuffer`) or directly mmapped from a file. Conversely, we can consume a `NodeTree`, extracting the bytes that express what has been added to the immutable part, together with the original immutable part. This gives callers the choice to start a new Nodetree. After writing to disk, some would prefer to reread for best guarantees (very cheap if mmapping), some others will find it more convenient to grow the memory that was considered immutable in the `NodeTree` and continue from there. This is enough to build examples running on real data and start gathering performance hints. Differential Revision: https://phab.mercurial-scm.org/D7796

pyoxidizer: allow extensions to be loaded from the file system It seems that setting this config is all that's needed to be able to load extensions from the file system (which we clearly want). Thanks for making this work, Gregory Szorc!. Differential Revision: https://phab.mercurial-scm.org/D8122