test-nointerrupt: make "sure" the handler "might" trigger (issue6558) We are sure that the signal got sent in the right time frame, however, we still have race, so either the code is actually buggy or we need some security to make sure the signal get processed. We might be affected by https://bugs.python.org/issue43406 ? Differential Revision: https://phab.mercurial-scm.org/D11251

testing: make sure write_file is "atomic" This make sure viewer cannot see the new file with partial content. This was likely the cause of some flakiness in `test-nointerrupt.t` Differential Revision: https://phab.mercurial-scm.org/D11250

test: disable test-subrepo-git.t in python2 + chg I am a couple of days in try to debug that at it seems minor enough with enough other priority to simply disable it for now. Differential Revision: https://phab.mercurial-scm.org/D11253

hgwebdir: avoid systematic full garbage collection Forcing a systematic full garbage collection upon each request can serioulsy harm performance. This is reported as https://bz.mercurial-scm.org/show_bug.cgi?id=6075 With this change we're performing the full collection according to a new setting, `experimental.web.full-garbage-collection-rate`. The default value is 1, which doesn't change the behavior and will allow us to test on real use cases. If the value is 0, no full garbage collection occurs. Regardless of the value of the setting, a partial garbage collection still occurs upon each request (not attempting to collect objects from the oldest generation). This should be enough to take care of reference cycles that have been created by the last request (assessment of this requires changing the setting, not to be 1). In my experience chasing memory leaks in Mercurial servers, the full collection never reclaimed any memory, but this is with Python 3 and biased towards small repositories. On the other hand, as explained in the Python developer docs [1], frequent full collections are very harmful in terms of performance if lots of objects survive the collection, and hence stay in the oldest generation. Note that `gc.collect()` is indeed trying to collect the oldest generation [2]. This happens usually in two cases: - unwanted lingering objects (i.e., an actual memory leak that the GC cannot do anything about). Sadly, we have lots of those these days. - desireable long-term objects, typically in caches (not inner caches carried by repositories, which should be collected with them). This is a subject of interest for the Heptapod project. In short, the flat rate that this change still permits is probably a bad idea in most cases, and the default value can be tweaked later on (or even be set to 0) according to experiments in the wild. The test is inspired from test-hgwebdir-paths.py [1] https://devguide.python.org/garbage_collector/#collecting-the-oldest-generation [2] https://docs.python.org/3/library/gc.html#gc.collect Differential Revision: https://phab.mercurial-scm.org/D11204

check-module-imports: ignore non-stdlib module installed by distribution Previously, the check script would detect breezy as part of the stdlib if installed using the debian package manager. This silence the following complains: tests/test-convert-bzr.t:117: imports not lexically sorted: breezy.bzr.bzrdir < sys tests/test-convert-bzr.t:117: stdlib import "breezy.bzr.bzrdir" follows local import: breezy tests/test-convert-bzr-ghosts.t:7: imports not lexically sorted: breezy.bzr.bzrdir < sys tests/test-convert-bzr-ghosts.t:7: stdlib import "breezy.bzr.bzrdir" follows local import: breezy tests/test-convert-bzr-treeroot.t:7: imports not lexically sorted: breezy.bzr.bzrdir < sys tests/test-convert-bzr-treeroot.t:7: stdlib import "breezy.bzr.bzrdir" follows local import: breezy Differential Revision: https://phab.mercurial-scm.org/D11249

remotefilelog: fix what looks like a wrong refactoring when various store functions started returning a revlog type as the first element of the tuple. Differential Revision: https://phab.mercurial-scm.org/D11243

rust-nodemap: falling back to C impl as mitigation This is a mitigation for https://bz.mercurial-scm.org/show_bug.cgi?id=6554 We see sometimes almost all data except the most recent revisions removed from the persistent nodemap, but we don't know how to reproduce yet. This has sadly repercussions beyond just needing to reconstruct the persistent nodemap: for instance, this automatically filters out all bookmarks pointing to revisions that the nodemap cannot resolve. If such filtering happens in a transaction, the update of the bookmarks file that happens at the end of transaction loses all bookmarks that have been affected. There may be similar consequences for other data. So this is a data loss, something that we have to prevent as soon as possible. As a mitigation measure, we will now fallback to the C implementation in case nodemap lookups failed. This will add some latency, e.g., in discovery, yet less than disabling the persistent nodemap entirely. We considered implementing the fallback directly on the Python side, but `revlog.get_rev()` is not systematically used, there are also several direct calls to the index method (`self.index.rev()` for a `revlog` instance). It is therefore more direct to implement the mitigation in the rust-cpython wrapper. Differential Revision: https://phab.mercurial-scm.org/D11238

obsolete: disable other evolution config options if createmarkers is off We used to raise an abort in this case, but recent changes to local clone command (377d8fc20e34) resulted in destrepo both caring about experimental.evolution config options and not initializing extensions. So imagine if you had evolve and allowdivergence enabled in your ~/.hgrc. Local clone stopped working after 377d8fc20e34 because evolve sets experimental.evolution=all, but only on srcrepo, for destrepo the extension is not initialized. It's possible to make local cloning work by initializing extensions for destrepo in some cases, but in other cases (e.g. allowdivergence in ~/.hgrc, evolve extension in original-repo/.hg/hgrc) it would still fail. In a discussion with Pierre-Yves David it was decided to simply force other evolution options to be false if createmarkers is not enabled. Differential Revision: https://phab.mercurial-scm.org/D11223

fix: use obsolete.isenabled() to check for experimental.allowdivergence Now that obsolete.isenabled() can also check if divergence is allowed, let's use it for consistency. Other experimental.evolution options are already checked via this function. Differential Revision: https://phab.mercurial-scm.org/D11222

rebase: use obsolete.isenabled() to check for experimental.allowdivergence Now that obsolete.isenabled() can also check if divergence is allowed, let's use it for consistency. Other experimental.evolution options are already checked via this function. Differential Revision: https://phab.mercurial-scm.org/D11221