sparse: move pruning of temporary includes into core This was our last method on the custom repo type, meaning we could remove that custom type and inline the 2 lines of code into reposetup(). As part of the move, instead of wrapping merge.update() from the sparse extension, we inline the function call. The ported function now no-ops if sparse isn't enabled, making it safe to always call. The call site in update() may not be the most appropriate. But it matches the previous behavior, which is the safest thing to do. It can be improved later.

sparse: move function for resolving sparse matcher into core As part of the move, the function arguments changed so revs are passed as a list instead of *args. This allows us to use keyword arguments properly. Since the plan is to integrate sparse into core and have it enabled by default, we need to prepare for a sparse matcher to always be obtained and operated on. As part of the move, we inserted code that returns an always matcher if sparse isn't enabled. Some callers in the sparse extension take this into account and conditionally perform matching depending on whether the special always matcher is seen. I /think/ this may have sped up some operations where the extension is installed but no sparse config is activated. One thing I'm ensure of in this code is whether os.path.dirname() is semantically correct. os.posixpath.dirname() (which is exported as pathutil.dirname) might be a better choise because all patterns should be using posix directory separators (/) instead of Windows (\). There's an inline comment that implies Windows was tested. So hopefully it won't be a problem. We can improve this in a follow-up. I've added a TODO to track it.

match: move matchers from sparse into core The sparse extension contains some matcher types that are generic and can exist in core. As part of the move, the classes now inherit from basematcher. always(), files(), and isexact() have been dropped because they match the default implementations in basematcher.

sparse: clean up config signature code Before, 0 was being used as the default signature value and we cast the int to a string. We also handled I/O exceptions manually. The new code uses cfs.tryread() so we always feed data into the hasher. The empty string does hash and and should be suitable for input into a cache key. The changes made the code simple enough that the separate checksum function could be inlined.

sparse: move config signature logic into core This is a pretty straightforward port. It will be cleaned up in a subsequent commit.

sparse: remove custom hash matcher With the recent change to always use repr(), this function was functionally identical to the version in fsmonitor it was replacing. So remove it.

sparse: override __repr__ in matchers sparse.py in FB's hg-experimental repo switched to using __repr__ for non-sparse matchers soon after hg core started overriding __repr__ in the matchers in match.py (because the core matchers also stopped having "includepat" and other attributes that sparse used to depend on). Let's finish that migration by implementing __repr__ in the sparse matchers as well. That also lets us remove the special handling of them in _hashmatcher().

tests: fix reference to undefined variable The delaypush() function had a reference to "repo" that was clearly supposed to be "pushop.repo". Instead of just fixing that, let's extract "pushop.repo.ui" to a variable, since that's the only piece of the repo that's needed in the function. I have not looked into why I saw a different result in the test to start with, but that's for another patch anyway.

shelve: don't reimplement mergestate.unresolved()

summary: don't reimplment mergestate.unresolved()

mergestate: implement unresolvedcount() in terms of unresolved() This simplifies the method slightly. It does create a full list of paths while doing so, but it's not a lot of data anyway (besides, I would think references to strings are no larger than (references to?) True).

mergestate: make unresolved() use iteritems() mergestate.unresolved() is a generator, so it seems better for it to rely on iteritems() than items(), although it also seems unlikely for it to make a noticeable difference.

changegroup: don't fail on empty changegroup (API) I don't know why applying an empty changegroup should be an error. It seems harmless. I suspect the check was there to find code that creates empty changegroups just because that would be wasteful. Let's use develwarn() for that instead, so we catch any such cases that run with our test runner, but we still allow others to generate empty changegroups if they want to. We have run into this check at Google once or twice and had to work around it, but I'm changing this not so much because of that, but because it seems like it shouldn't be an error. I also changed the message slightly to be more modern ("changelog group" -> "changegroup") and more generic ("received" -> "applied").

changegroup: remove option to allow empty changegroup (API) No caller sets the "emptyok" option, so let's remove it.

strip: don't allow empty changegroup in bundle1 Applying an empty changegroup has been an error since the beginning. The only exception was strip, which would allow to apply an empty changegroup from the temporary bundle. However, the emptyok=True option was only set for bundle1 bundles. In other words, temporary bundle2 bundles would fail if they were empty. Bundle2 has now been used enough that it seems safe to say that we simply don't create bundle2 bundles with empty changegroups. That also suggests that we never create bundle1 bundles with empty changegroups (i.e. empty bundle1 bundles, since bundle1 is just a changegroup), because, AFAICT, the code leading up to the application of the bundle is the same for bundle1 and bundle2. Therefore, let's stop passing emptyok=True, so we more clearly get the same behavior for bundle1 and bundle2.

match: minor cleanups to patternmatcher and includematcher The "patterns"/"include" in "patternspat"/"includepat" is redundant, so drop it. Also a "_" prefix since it's "private". Inline the "pm"/"im" variables.

py3: fix test-diff-newlines.t to be compatible with py3

sparse: move some temporary includes functions into core Functions for reading and writing the tempsparse file have been moved. prunetemporaryincludes() will be moved separately because it is non-trivial.

sparse: move config file writing into core The code was refactored during the move to be more procedural instead of using string formatting. This has the benefit of not writing empty sections, which changed tests.

localrepo: add sparse caches The sparse extension maintains caches for the sparse files to a signature and a signature to a matcher. This allows the sparse matchers to be resolved quickly, which is apparently something that can occur in loops. This patch ports the sparse caches to the localrepo class pretty much as-is. There is potentially room to improve the caching mechanism. But that can be done as a follow-up. The default invalidatecaches() now clears the relevant sparse cache. invalidatesignaturecache() has been moved to sparse.py.

sparse: move active profiles function into core Also includes some light formatting changes.

sparse: move resolving of sparse patterns for rev into core This method is reasonably well-contained and simple to move. As part of the move, some light formatting was performed. A "working copy" reference in an error message was changed to "working directory." The biggest change was to _refreshoncommit() in sparse.py. It was previously checking for the existence of an attribute on the repo instance. Since the moved function now returns empty data if sparse isn't enabled, we unconditionally call the new function. However, we do have to protect another method call in that function. This will all be unhacked eventually.

sparse: variable to track if sparse is enabled Currently, the sparse extension sniffs repo instances for attributes defined by the sparse extension to determine if sparse is enabled. As we move code away from repo instances, these checks will be a bit more brittle. We introduce a module-level variable to track whether sparse is enabled as a temporary workaround.

sparse: move profile reading into core One more step towards weaning off methods on repo instances and moving code to core. While this function is only used once and is simple, it needs to exist on its own so Facebook can monkeypatch it to enable simplecache integration.

sparse: move config parsing into core This patch marks the beginning of moving code from the sparse extension into core. The goal is to move as much of the functionality as possible into core, where it will be an experimental feature. The extension will likely continue to exist to enable the feature and provide UI elements. As part of the move, the repo method was converted to a module function. It doesn't need to exist on repos. An error message was also updated to reflect that an error isn't necessarily from the .hg/sparse file. The API should be updated later to pass in a filename so the error can be more descriptive. Copyright of the added file was copied from the sparse extension.

sparse: use vfs.tryread() vfs.exists() followed by a file read is an anti-pattern because it incurs an extra stat() to test for file presence. vfs.tryread() returns empty string on missing file and avoids the stat().

sparse: refactor sparsechecksum() This was relying on garbage collection to close the opened file, which is a bug. Both callers simply called into self.vfs to resolve the path. So refactor to use the vfs layer. While we're here, rename the method to reflect it is internal and to break anyone relying on the old behavior.

sparse: document config file format This was previously undocumented. Seems useful to have.

sparse: rename command to debugsparse Sparse checkout is still highly experimental and not protected by BC guarantees yet. We also haven't had a discussion on the UX. To discourage use, we rename the sparse command to debugsparse.

sparse: remove reference to simplecache This is a 3rd party extension authored by Facebook. References in core are not appropriate. It will be possible to restore this code/optimization via monkeypatching. So Facebook won't lose any functionality. The removed code is important for performance. So add a comment tracking it.

sparse: remove reference to hgwatchman This is a legacy extension. Now that the extension is in core, we only need to support what's in core, which is fsmonitor.

sparse: expand module docstring Clarify lack of BC guarantees. And say a bit more about the extension.

sparse: vendor Facebook-developed extension Facebook has developed an extension to enable "sparse" checkouts - a working directory with a subset of files. This feature is a critical component in enabling repositories to scale to infinite number of files while retaining reasonable performance. It's worth noting that sparse checkout is only one possible solution to this problem: another is virtual filesystems that realize files on first access. But given that virtual filesystems may not be accessible to all users, sparse checkout is necessary as a fallback. Per mailing list discussion at https://www.mercurial-scm.org/pipermail/mercurial-devel/2017-March/095868.html we want to add sparse checkout to the Mercurial distribution via roughly the following mechanism: 1. Vendor extension as-is with minimal modifications (this patch) 2. Refactor extension so it is more clearly experimental and inline with Mercurial practices 3. Move code from extension into core where possible 4. Drop experimental labeling and/or move feature into core after sign-off from narrow clone feature owners This commit essentially copies the sparse extension and tests from revision 71e0a2aeca92a4078fe1b8c76e32c88ff1929737 of the https://bitbucket.org/facebook/hg-experimental repository. A list of modifications made as part of vendoring is as follows: * "EXPERIMENTAL" added to module docstring * Imports were changed to match Mercurial style conventions * "testedwith" value was updated to core Mercurial special value and comment boilerplate was inserted * A "clone_sparse" function was renamed to "clonesparse" to appease the style checker * Paths to the sparse extension in tests reflect built-in location * test-sparse-extensions.t was renamed to test-sparse-fsmonitor.t and references to "simplecache" were removed. The test always skips because it isn't trivial to run it given the way we currently run fsmonitor tests * A double empty line was removed from test-sparse-profiles.t There are aspects of the added code that are obviously not ideal. The goal is to make a minimal number of modifications as part of the vendoring to make it easier to track changes from the original implementation. Refactoring will occur in subsequent patches.

contrib: widen "direct use of `python`" net again I think I've now caught all of them. Differential Revision: https://phab.mercurial-scm.org/D15

tests: clean up a newly-introduced instance of `python` Differential Revision: https://phab.mercurial-scm.org/D16

tests: clean up even more direct `python` calls with $PYTHON This time ones that are prefixed with =, ", ', or `. This appears to be the last of them. Differential Revision: https://phab.mercurial-scm.org/D14

contrib: widen the "don't use `python`" net a little I'm still cleaning this up, but it's easier to do in bite-size chunks like this than all at once. The negative lookahead avoids one false positive category from some output related to finding Subversion bindings. Differential Revision: https://phab.mercurial-scm.org/D13

followlines: join merge parents line ranges in blockdescendants() (issue5595) In blockdescendants(), we had an assertion when line range of a merge changeset was not consistent depending on which parent was considered for computation. For instance, this might occur when file content (in lookup range) is significantly different between parent branches of the merge as demonstrated in added tests (where we almost completely rewrite the "baz" file while also introducing similarities with its content in the other branch we later merge to). Now, in such case, we combine line ranges from all parents by storing the envelope of both line ranges. This is conservative (the line range is extended, possibly unnecessarily) but at least this should avoid missing descendants with changes in a range that would fall in that of one parent but not in another one (the case of "baz: narrow change (2->2+)" changeset in tests).

workingfilectx: add exists, lexists Switch the lone call in merge.py to use it. As with past refactors, the goal is to make wctx hot-swappable with an in-memory context in the future. This change should be a no-op today.

vfs: add explanation about cost of checkambig=True in corner case

vfs: replace avoiding ambiguity in abstractvfs.rename with _avoidambig This centralizes common logic to forcibly avoid file stat ambiguity into _avoidambig(), which was introduced by previous patch.

vfs: copy if EPERM to avoid file stat ambiguity forcibly at closing Now, files (to be truncated) are opened with checkambig=True, only if localrepository caches it. Therefore, straightforward "copy if EPERM" is always reasonable to avoid file stat ambiguity at closing. This patch makes checkambigatclosing close wrapper copy the target file, and advance mtime on it after renaming, if EPERM. This can avoid file stat ambiguity, even if the target file is owned by another (see issue5418 and issue5584 for detail). This patch factors main logic out instead of changing checkambigatclosing._checkambig() directly, in order to reuse it.

transaction: apply checkambig=True only on limited files for similarity Now, transaction can determine whether avoidance of file stat ambiguity is needed for each files, by blacklist "checkambigfiles". For similarity to truncation in _playback(), this patch apply checkambig=True only on limited files in code paths below. - restoring files by util.copyfile(), in _playback() (checkambigfiles itself is examined at first, because it as a keyword argument might be None) - writing files at finalization of transaction, in _generatefiles() This patch reduces cost of checking stat at writing out and restoring files, which aren't filecache-ed.

transaction: avoid file stat ambiguity only for files in blacklist Advancing mtime by os.utime() fails for EPERM, if the target file is owned by another. bff5ccbe5ead and related changes made some code paths give advancing mtime up in such case, to fix issue5418. This causes file stat ambiguity (again), if it is owned by another. https://www.mercurial-scm.org/wiki/ExactCacheValidationPlan To avoid file stat ambiguity in such case, especially for .hg/dirstate, ed66ec39933f made vfs.rename() copy the target file, and advance mtime of renamed one again, if EPERM (see issue5584 for detail). But straightforward "copy if EPERM" isn't reasonable for truncation of append-only files at rollbacking, because rollbacking might cost much for truncation of many filelogs, even though filelogs aren't filecache-ed. Therefore, this patch introduces blacklist "checkambigfiles", and avoids file stat ambiguity only for files specified in this blacklist. This patch consists of two parts below, which should be applied at once in order to avoid regression. - specify 'checkambig=True' at vfs.open(mode='a') in _playback() according to checkambigfiles - invoke _playback() with checkambigfiles - add transaction.__init__() checkambigfiles argument, for _abort() - make localrepo instantiate transaction with _cachedfiles - add rollback() checkambigfiles argument, for "hg rollback/recover" - make localrepo invoke rollback() with _cachedfiles After this patch, straightforward "copy if EPERM" will be reasonable at closing the file opened with checkambig=True, because this policy is applied only on files, which are listed in blacklist "checkambigfiles".

localrepo: store path and vfs location of cached properties This information is used to make transaction handle these files specially, in order to avoid file stat ambiguity of them. Gathering information about cached files via annotation classes can avoid overlooking properties newly introduced in the future.

template: add successors template Add a 'successorssets' template that returns the list of all closest known sucessorssets for a changectx. The elements of the list are changesets. The "closest successors" are the first locally known revisions encountered while, walking successors markers. It uses successorsets previously modified to support the closest argument. This logic respect repository filtering. So hidden revision will be skipped by this logic unless --hidden is specified. Since we only display the visible predecessors, this template will not display anything in most case. It makes a good candidate for inclusion in the default log output. I updated the test-obsmarker-template.t test file introduced with the predecessors template to test successorssets template.

template: add tests for more complex cases We add new tests for improving the coverage of existing obs-markers related template (predecessors) and the new one we are introducing (successorssets).

obsolete: closest divergent support Add a closest argument to successorssets changing the definition of latest successors. With "closest=false" (current behavior), latest successors are "leafs" on the obsmarker graph. They don't have any successor and are known locally. With "closest=true", latest successors are the closest locally-known changesets that are visible in the repository or repoview. Closest successors can be then obsolete, orphan. This will be used in a later patch to show the closest successor of changesets with the successorssets template.

obsolete: pass cache argument of successors set explicitly We plan to add a new argument to successorsets. But first we need to update all callers to pass cache argument explicitly to avoid arguments confusion.

obsolete: small doc update for 'successorssets' Clarify successorssets documentation before we start updating the main function. This patch serie will introduce the successorssets template, the opposite of predecessor templates. Successors will use successorssets function and requires some improvement so before doing that, we clean up successorssets a bit.

phabricator: do not read a same revision twice It's possible to set up non-linear dependencies in Phabricator like: o D4 |\ | o D3 | | o | D2 |/ o D1 The old `phabread` code will print D1 twice. This patch adds de-duplication to prevent that. Test Plan: Construct the above dependencies in a Phabricator test instance and make sure the old code prints D1 twice while the new code won't.

patch: make parsepatch optionally trim context lines Previously there is a suspicious `if False and delta > 0` which dates back to the beginning of hgext/record.py (b2607267236d). The "trimming context lines" feature could be useful (and is used by the next patch). So let's enable it. This patch adds a new `maxcontext` parameter to `recordhunk` and `parsepatch`, changing the `if False` condition to respect it. The old `trimcontext` implementation is also wrong - it does not update `toline` correctly and it does not do the right thing for `before` context. A doctest was added to guard us from making a similar mistake again. Since `maxcontext` is set to `None` (unlimited), there is no behavior change.

phabricator: try to fetch differential revisions in batch Previously, we read Differential Revisions one by one by calling `differential.query`. Fetching them one by one is suboptimal. Unfortunately, there is no Conduit API that allows us to get a stack of diffids using a single API call. This patch tries to be smarter using a simple heuristic: when fetching D59 as a stack, previous IDs like D51, D52, D53, ..., D58 are likely belonging to a same stack so just fetch them as well. Since `differential.query` only returns cheap metadata without expensive diff content, it shouldn't be a big problem for the server. Using a test Phabricator instance, this patch reduces `phabread` reading a 10 patch stack from about 13 to 30 seconds to 8 seconds.

phabricator: avoid calling differential.getcommitmessage Previously, we call differential.getcommitmessage API to get commit messages. Now we read that from "Differential Revision" object fetched via "differential.query" API. This removes one API call per patch.

phabricator: rework phabread to reduce memory usage and round-trips This patch reworked phabread a bit so it fetches the lightweight "Differential Revision" metadata for a stack first. Then read other data. This allows the code to: a) send 1 request to get all `hg:meta` data instead of N requests b) patches are read in desired order, no need to buffer the output "b)" reduces the memory usage from O(N^2) to O(N) since we no longer keep old patch contents in memory. The above `N` means the number of patches in the stack.

phabricator: abort if phabsend gets empty revs Previously we didn't abort. Now we abort if revs is empty. This is consistent with "hg export" behavior. Maybe "return 1" is also a reasonable behavior, but that's inconsistent with the existing "hg export".

phabricator: do not upload new diff if nothing changes Previously, `phabsend` uploads new diffs as long as the commit hash changes. That's suboptimal because sometimes the diff is exactly the same as before, the commit hash change is caused by a parent hash change, or commit message change which do not affect diff content. This patch adds a check examining actual diff contents to skip uploading new diffs in that case.

phabricator: add node and p1 to hg:meta property The "hg:meta" property is for extra metadata to reconstruct the patch. Previously it does not have node or parent information since I think by reading the patch again, the commit message will be mangled (like, added the "Differential Revision" line) and we cannot preserve the commit hash. However, the "parent" information could be useful. It could be helpful to locate the "base revision" so in case of a conflict applying the patch directly, we might be able to use 3-way merge to resolve it correctly. Note: "local:commits" is an existing "property" used by Phabricator that has the node and parent information. However, it lacks of timezone information and requires "author" and "authorEmail" to be separated. So we are using a different "property" - "hg:meta" to be distinguished from "local:commits".

phabricator: check associated Differential Revision from commit message Previously, only tags can "associate" a changeset to a Differential Revision. But the usual pattern (arc patch or hg phabread) is to put the Differential Revision URL in commit message. This patch makes the code read commit message to find associated Differential Revision if associated tags are not found. This makes some workflows possible. For example, if the author loses their repo, or switch to another computer, they can continue download their own patches from Phabricator and update them without needing to manually create tags.

tests: replace yet more calls to `python` with $PYTHON These are some simple cases. More to come in a future change. Reviewers: krbullock Reviewed By: krbullock Differential Revision: https://phab.mercurial-scm.org/D4

tests: capitalize Python when it's not used as a command name This avoids some false positives in an upcoming check-code rule. Reviewers: krbullock Reviewed By: krbullock Differential Revision: https://phab.mercurial-scm.org/D3

merge with stable

vfs: drop the 'mustaudit' API There are no remaining users of 'mustaudit' so we can safely drop the API. External user are unlikely from a quick research so no deprecation is added.

streamclone: stop using 'vfs.mustaudit = False' Now that each call disable the auditing on its own, we can safely drop this the mustaudit usage. No other code is modified.

vfs: simplify path audit disabling in stream clone The whole 'mustaudit' API is quite complex compared to its actual usage by its unique user in stream clone. Instead we add a "auditpath" parameter to 'vfs.__call_'. The stream clone code then explicitly open files with path auditing disabled. The 'mustaudit' API will be cleaned up in the next changeset.

auditor: add simple comment about repo.auditor and al Every once in a while, I get confused by what these are. Let us add a comment.

tag: make sure the repository is locked when tagging Otherwise, writing localtag can happen without the lock.

test: glob a line number in test-devel-warnings.t This make is simpler to edit the extensions file without side effect.

test: add a small comment to explain a section of test-devel-warning This makes each test boundaries clearer.

obsolete: reports the number of local changeset obsoleted when unbundling This is a first basic visible usage of the changes tracking in the transaction. We adds a new function computing the pre-existing changesets obsoleted by a transaction and a transaction call back displaying this information. Example output: added 1 changesets with 1 changes to 1 files (+1 heads) 3 new obsolescence markers obsoleted 1 changesets The goal is to evolve the transaction summary into something bigger, gathering existing output there and adding new useful one. This patch is a good first step on this road. The new output is basic but give a user to the content of tr.changes['obsmarkers'] and give an idea of the new options we haves. I expect to revisit the message soon. The caller recording the transaction summary should also be moved into a more generic location but further refactoring is needed before it can happen.

transaction: track new obsmarkers in the 'changes' mapping The obsstore collaborate with transaction to make sure we track all the obsmarkers added during a transaction. This will be useful for various usages: hooks, caches, better output, etc. This is the seconds kind of data added to tr.changes (first one was added revisions)

configitems: register the 'progress.estimate' config

configitems: register the 'progress.clear-complete' config

configitems: register the 'progress.assume-tty' config

configitems: register the 'format.usestore' config

configitems: register the 'format.usegeneraldelta' config

configitems: register the 'format.usefncache' config

configitems: register the 'format.obsstore-version' config

configitems: register the 'factotum.service' config

configitems: register the 'factotum.mountpoint' config

configitems: register the 'factotum.executable' config

configitems: register the 'format.maxchainlen' config

configitems: register the 'format.manifestcachesize' config

configitems: register the 'format.generaldelta' config

configitems: register the 'format.dotencode' config

configitems: register the 'format.chunkcachesize' config

configitems: register the 'format.aggressivemergedeltas' config

configitems: gather comment related to 'worker.backgroundclosemaxqueue' Thanks to Yuya for pointing this out.

configitems: register the 'worker.numcpus' config

configitems: register the 'worker.backgroundclosethreadcount' config

configitems: register the 'worker.backgroundcloseminfilecount' config

configitems: register the 'worker.backgroundclosemaxqueue' config

configitems: register the 'patch.eol' config

configitems: register the 'server.zliblevel' config

configitems: register the 'server.validate' config

configitems: register the 'server.uncompressedallowsecret' config

configitems: register the 'server.preferuncompressed' config

configitems: register the 'server.maxhttpheaderlen' config

configitems: register the 'server.disablefullbundle' config

configitems: register the 'server.concurrent-push-mode' config

configitems: register the 'server.compressionengines' config

configitems: register the 'server.bundle1gd' config

configitems: register the 'server.bundle1' config

configitems: register the 'hostsecurity.disabletls10warning' config

configitems: register the 'hostsecurity.ciphers' config

configitem: create a new list of each 'acl.sources' access Thanks for goes to Yuya for spotting this.

dirstate: centralize _cwd handling into _cwd method Before this patch, immediate value is assigned to dirstate._cwd, if ui.forcecwd is specified at instantiation of dirstate. But this doesn't work as expected in some cases. For example, hgweb set ui.forcecwd after instantiation of repo object. If an extension touches repo.dirstate in its reposetup(), dirstate is instantiated without setting ui.forcecwd, and dirstate.getcwd() returns incorrect result. In addition to it, hgweb.__init__() can take already instantiated repo object, too. In this case, repo.dirstate might be already instantiated, even if all enabled extensions don't so in their own reposetup(). To avoid such issue, this patch centralizes _cwd handling into _cwd method. This issue can be reproduced by running test-hgweb-commands.t with fsmonitor-run-tests.py.

tests: add line specific for testing with fsmonitor

tests: make output lines conditional for testing with fsmonitor Repository cloned-bookmark-default and tobundle exist in the working directory of main test repository "repo". We should take care for them, because it is known issue that fsmonitor can't handle nested repositories. These nested repositories are cloned from "repo", and the number of unknown files = files in these repositories (including files under .hg) will be changed easily in the future. But testing with fsmonitor is not ordinary. Therefore, test-bookmarks.t with fsmonitor might be broken silently. This is reason why this patch uses "(glob)" for the number of unknown files in "hg summary" output. BTW, this patch doesn't use .hgignore to make test portable, because .hgignore might cause another issue related to "walk_on_invalidate" configuration of fsmonitor.

tests: add fsmonitor specific output lines at enabling largefiles Temporarily enabling largefiles causes these output lines, only if tests are executed with fsmonitor.

show: document why accidentally quadratic is (probably) acceptable

show: also catch AmbiguousCommand cmdutil.findcmd() can raise this as well. While we'll almost certainly never encounter this in the wild, guard against it regardless.

show: avoid extra list operations

show: tweak warning message '.' is "working directory parent" not "working directory."

revlog: add an experimental option to mitigated delta issues (issue5480) The general delta heuristic to select a delta do not scale with the number of branch. The delta base is frequently too far away to be able to reuse a chain according to the "distance" criteria. This leads to insertion of larger delta (or even full text) that themselves push the bases for the next delta further away leading to more large deltas and full texts. This full text and frequent recomputation throw Mercurial performance in disarray. For example of a slightly large repository 280 000 files (2 150 000 versions) 430 000 changesets (10 000 topological heads) Number below compares repository with and without the distance criteria: manifest size: with: 21.4 GB without: 0.3 GB store size: with: 28.7 GB without 7.4 GB bundle last 15 00 revisions: with: 800 seconds 971 MB without: 50 seconds 73 MB unbundle time (of the last 15K revisions): with: 1150 seconds (~19 minutes) without: 35 seconds Similar issues has been observed in other repositories. Adding a new option or "feature" on stable is uncommon. However, given that this issues is making Mercurial practically unusable, I'm exceptionally targeting this patch for stable. What is actually needed is a full rework of the delta building and reading logic. However, that will be a longer process and churn not suitable for stable. In the meantime, we introduces a quick and dirty mitigation of this in the 'experimental' config space. The new option introduces a way to set the maximum amount of memory usable to store a diff in memory. This extend the ability for Mercurial to create chains without removing all safe guard regarding memory access. The option should be phased out when core has a more proper solution available. Setting the limit to '0' remove all limits, setting it to '-1' use the default limit (textsize x 4).

tests: use system hg only if changelog or dirstate can't be read The bundled hg should work flawlessly in most cases. Make it depend on the external installation only if necessary since we can't control the whole environment. This patch doesn't implement the "exit 80" idea proposed by Jun. I don't want to keep the capability checking sync with the actual tests.

tests: restore workaround of obsolete warning from 3c9066ed557c It's simple and works well unless you are using third-party extensions that changes the store format.

tests: alias syshg and syshgenv so they can be switched conditionally

tests: actually restore the original environment before running syshg Since os.environ may be overridden in run-tests.py, several important variables such as PATH weren't restored. I don't like the idea of using the system hg *by default* because the executable and the configs are out of our control. But I don't mind as long as the tests pass.

Added signature for changeset 26c49ed51a69