manifest: duplicate call to addrevision() When we start writing submanifests to their own revlogs, we will not want to write a new revision for a directory if there were no changes to it. To prepare for this, duplicate the call to addrevision() and move them earlier where they can more easily be avoided.

record: add message when starting record's curses interface We are adding this log message to reduce a confusion when a command prints something just before starting the curses interface. Since the interactive mode is taking over the entire screen, starts with no delay and does wait for a key press, the user believes that messages printed before opening the interactive mode were actually printed after using interactive mode, not before. The fix adds the line "Starting interactive mode" helping the user separate the messages that were printed before and after the start of the interactive mode. One particular example where this was a problem is the revert command where we first print the list of changes to be considered for revert, then opens the curses interface right away without letting the user see the messages. The user then selects the changes, validates and then see the messages from before opening the interactive mode and is confused.

subrepo: add include/exclude support for diffing git subrepos Previously, git subrepos were ignored if any type of path selection was done. This can be solved by using subrepo status and filtering matching files.

revset: undocument wdir() until its command outputs get stable wdir() implementation is still incomplete and shouldn't be advertised to users. This patch will be backed out when - template values such as {rev} and {node} are settled - major commands and revsets work without crashing

linkrev: fix issue with annotate of working copy The introrev was appearing as None in new annotate tests, which the code from the stable branch wasn't expecting.

merge with stable

histedit: delete all non-actionclass related code Now that all actions have been migrated to the class format, we can delete all the unnecessary code that supported the old function format.

histedit: improve roll action integration with fold Previously the fold action would inspect it's class to figure out if it was a rollup or not. This was hacky. Now that finishfold is inside the fold class, let's modify it to check a function (which roll can override) to determine if it should be prompting for a commit message.

histedit: move finishfold into fold class Let's move the finishfold function into the fold class so we can modify it from the roll class more easily.

histedit: convert fold/roll actions into a class This converts the fold/roll actions into a histeditclass instance, as part of an ongoing effort to refactor histedit for maintainability and robustness. The tests changed for two reasons: 1) We get a new 'empty changeset' warning because we now warn more consistently between normal histedit and --continue about commits disappearing. 2) Previously we were not putting the histedit-source extra field on the temporary fold commit during normal runs, but we were on --continue runs. By unifying these code paths we now consistently put histedit-source on the temporary fold commit, which changes some of the hashes in the backup bundles.

histedit: convert edit action into a class This converts the edit action into a histeditclass instance, as part of an ongoing effort to refactor histedit for maintainability and robustness.

histedit: convert message action into a class This converts the message action into a histeditclass instance, as part of an ongoing effort to refactor histedit for maintainability and robustness.

histedit: convert drop action into a class This converts the drop action into a histeditclass instance, as part of an ongoing effort to refactor histedit for maintainability and robustness.

histedit: convert pick action into a class This converts the pick action into a histeditclass instance, as part of an ongoing effort to refactor histedit for maintainability and robustness. The test output changed because previously pick would only report the commit disappearing if there were no merge conflicts. Now that we've unified the normal and the --continue flows, they act the same and we get the warning even after --continue.

histedit: integrate action class into flow This takes the newly added histeditaction class and integrates it into the histedit run and bootstrapcontinue logic. No actions implement the class yet, but they will be add in upcoming patches.

histedit: add a new histeditaction class This adds a new class called histeditaction. It represents a single action in a histedit plan. Future patches will integrate it into the histedit flow, then convert each existing action in to the class form one by one. This is part of a larger refactor aimed at increasing histedit robustness, maintainability, and extensibility.

histedit: allow histedit --continue when not on a descendant (BC) Previously we would not allow --continue if the current working copy parent was not a descendant of the commit produced by the previous histedit step. There's nothing really blocking us from continuing the histedit in this situation, so let's stop aborting in this case. This is technically a BC change, but it is making things more forgiving so I think it's ok. In the future we will likely add an 'exec' action to histedit, which means the user can do whatever they want during the middle of a histedit (like perhaps calling 'hg update'), which means we'll need to support this case eventually anyway.

tags: explicitly log which tags cache file is being written We now have multiple tags cache files. Record exactly which file is being written. This should help aid debugging into performance issues with any single filter.

tags: write a separate tags cache file for unfiltered repos Since we changed the format of the tags cache, we should bump the filename. Before this patch, "tags" was being used for unfiltered repositories. Change the naming scheme to be consistent and ensure that a new cache file is used. While I was here, I updated the docs to describe the existence of multiple caches. I also added explicit test coverage for the creation of the unfiltered tags cache.

tags: return empty list of heads for no .hgtags case The caller only uses the heads to resolve tags from content of .hgtags. Returning a non-empty array is pointless if there is no .hgtags file.

tags: change format of tags cache files .hgtags fnodes are now written to a shared cache file. They don't need to exist in the per-filter tags cache files. Stop writing them. The format of the tags cache file has changed in a backwards incompatible way. This should be acceptable, as we just established per-filter tags cache files and no client should have per-filter tags cache files that will need to be read. So no backwards compatbility concern is present. The new format has a single header line followed by resolved tags entries. The header line is similar to the old first line with a major difference: we now compute and store a hash of the filtered revisions. Before, if the set of filtered revs changed, we may return incorrect results. We now detect that. A test for verifying filtered rev change is handled properly has been added.

tags: don't read .hgtags fnodes from tags cache files Now that we have a standalone and shared cache for .hgtags fnodes, the .hgtags fnodes stored in the tags cache files are redundant. Upcoming patches will change the format of the tags cache files to remove this data and to improve the validation. To prepare for this, we change the tags reading code to ignore all but the tip .hgtags fnodes entries in existing tags cache files. All fnodes lookups now go through our new shared cache, which is why test output changed. Format of tags cache files has not yet changed.

rebase: restore bookmark state on abort The bookmark state was already being preserved, but it wasn't being properly restored.

histedit: store backup file before histedit It's possible for the user to delete some of the commits they started with during a histedit, and aborting the histedit doesn't bring them back. Let's store a backup bundle so we can always recover the stack of commits from before they began.

histedit: replace pickle with custom serialization Pickle is undesirable, so let's serialize it ourselves. We keep the ability to parse existing pickle blobs for now.

devel-warn: add a prefix to all messages ("devel-warn: ") We want to make the origin and importance of the message clear to developers.

push: acquire local 'wlock' if "pushback" is expected (BC) (issue4596) If the client allows "pushback", the bundle2 served back by the server may contains parts that will write to the repository. Such parts may require the 'wlock' (eg: bookmark) so we acquire it in advance to make sure it got acquired before the 'lock'.

annotate: always prepare ancestry context of base fctx (issue4600) This patch extends the workaround introduced by dd01834a696f. Even if the base fctx is the same as intorrev, _ancestrycontext must be built for faster _changeid lookup. repo: https://hg.mozilla.org/releases/mozilla-beta command: hg annotate -r 4954faa47dd0 gfx/thebes/gfxWindowsPlatform.cpp before: 52.450 sec after: 1.820 sec

unbundle: acquire 'wlock' when processing bundle2 (BC) (issue4596) A bundle2 may contain bookmark updates (or other extension content) that requires the 'wlock' to be written. As 'wlock' must be acquired before 'lock', we must stay on the side of caution and use both in all case to ensure their ordering.

run-test: enable the devel warning during tests This should help us to catch new locking order issues as soon as possible. There are two harmless test updates (from the config change). Moreover, some bundle2 tests are displaying warning for a legitimate reason. The use of pushkey during the unbundle process may requires the 'wlock' after 'lock' (around the whole unbundle process was taken). This is non-trivial to fix, so I better have the check on, with the warning in the test than the check off. See issue4596 for details.

wlock: do not warn for non-wait locking We are warning about lock acquired in the wrong order because this can create dead-lock situation. But non-wait acquisition will not block and therefore not create a dead-lock. So we do not need to wait in such case.

develwarn: include call site in the simple message version Just displaying the warning makes it quite hard to recognise the guilty code quickly and using --traceback for all calls is not very convenient. So we include the call site with all simple message to help developer to recognise errors sources.

develwarn: handle the end of line inside the function itself The traceback version should not have a end of line at all. The non-traceback version will requires the same things soon.

develwarn: refactor the developer warning logic The logic is currently duplicated and we plan to make it a bit smarter. So we move it into a function first to make the update more robust and simple.

lock: update the docstring with order information Lock must be acquired in a specific order to avoid dead-lock. This was documented on the wiki, but having this information in the docstring is also useful.

wlock: reword the devel warning We change the wording of the developer warning: - "lock" taken before "wlock" + "wlock" acquired after "lock" The goals here are to: - Put the "subject" as the first word, - use "acquired" instead of "taken" since it seems more accurate.

wlock: only issue devel warning when actually acquiring the lock Before this change, any call to 'wlock' after we acquired a 'lock' was issuing a warning. This is wrong as the 'wlock' have been properly acquired before the 'lock' in a previous call. We move the warning code to only issue such warnings when we are acquiring the 'wlock' -after- acquiring 'lock'. This is the expected behavior of this warning from the start.

bundle2: flush output in a part in all cases We want to preserve output even when the unbundling fails (eg: hook output). So we must make sure that everything we have is flushed into the reply bundle. (This is related to issue4594)

run-tests: also follow symlink when update PATH with 'run-tests.py' dir I'm using 'run-tests.py' from my '$PATH' and use a symlink to get 'run-tests.py' to in that '$PATH'. There is a handful of test helpers (like f) that needs to be in the '$PATH' for the test to run, and they are expected to live next to the 'run-tests.py' binary. Using a symlink confuses this logic, so we add to the '$PATH' both the 'run-tests.py' executable directory, and the actual file location direction.

bundle2: fix names for error part handler The name is not very important but copy paste banshee got there. We make distinctive name.

transaction: introduce a transaction ID, to be available to all hooks The transaction ID is built from the object ID and creation time stamp to make sure it is unique.

transaction: actually use tr.hookargs in pretxnclose The 'tr.hookargs' is a dictionary containing all the arguments to be passed to hooks. It is actually used for hooks now...

bundle2: use unbundle source as transaction name This is what a bundle 1 push is doing. This got caught by trying to run the whole test-suite using bundle2 for exchanges.

tags: have a different cache file per filter level Currently whichever filter level asks for tags last will write the data on disk. This create massive issues when tags are read for "visible" and "unfiltered" on large and multi headed repository (like Mozilla central). See issue4550 for details Each filter level recomputes its own cache without direct collaboration but they all share the same 'hgtagsfnodes' cache. And that is where most of the time is spent.

parsers: when available, use a presized dictionary for the file foldmap On a repo with over 300,000 files, this speeds up perffilefoldmap: before: wall 0.178421 comb 0.180000 user 0.160000 sys 0.020000 (best of 55) after: wall 0.164462 comb 0.160000 user 0.140000 sys 0.020000 (best of 59)

tags: extract .hgtags filenodes cache to a standalone file Resolution of .hgtags filenodes values has historically been a performance pain point for large repositories, where reading individual manifests can take over 100ms. Multiplied by hundreds or even thousands of heads and resolving .hgtags filenodes becomes a performance issue. This patch establishes a standalone cache file holding the .hgtags filenodes for each changeset. After this patch, the .hgtags filenode for any particular changeset should only have to be computed once during the lifetime of the repository. The introduced hgtagsfnodes1 cache file is modeled after the rev branch cache: the cache is effectively an array of entries consisting of a changeset fragment and the filenode for a revision. The file grows in proportion to the length of the repository (24 bytes per changeset) and is truncated when the repository is stripped. The file is not written unless tag info is requested and tags have changed since last time. This patch partially addresses issue4550. Future patches will split the "tags" cache file into per-filter files and will refactor the cache format to not capture the .hgtags fnodes, as these are now stored in the hgtagsfnodes1 cache. This patch is capable of standing alone. We should not have to wait on the tags cache filter split and format refactor for this patch to land.

extensions: extract partial application into a bind() function We use partial function application for wrapping existing Mercurial functions, and it's implemented separately each time. This patch extracts the partial application into a new bind() function that can be used on its own by extensions when appropriate. In particular, the evolve extension needs to wrap functions in the various bundle2 processing dictionaries, which the pre-existing methods don't support.

obsolete: experimental flag to get debug about obsmarkers exchange The obsolescence markers exchange is still experimental. We (developer) need more information about what is going on. I'm adding an experimental flag to add display the amount of data exchanged during bundle2 exchanges.

bundle2: add an 'idx' argument to the 'getbundle2partsgenerator' This argument let extensions control in what order bundle2 part are generated server side during a pull. This is useful to ensure the transaction is in a proper state before some actions or hooks happens.

bundle2: add an 'idx' argument to the 'b2partsgenerator' This argument let extensions control in what order bundle2 part are generated client side during a push. This is useful to ensure the transaction is in a proper state before some actions or hooks happens.

rpms: create missing builds dir if it doesn't exist

rpms: for packages with their own python, put it in /opt/python-hg This is more compliant with the FHS and Fedora packaging guidelines.

branchcache: stay silent if failing to read cache files The warning has in some cases incorrectly attributed unrelated problems to rbc. Instead, just do like the branch head cache does and stay quiet when reading fails. The cache will be missing the first time a repo is used. It is a normal situation and there is no reason to make a note of that.

merge: better debug messages before/after invoking external merge tool

subrepo: use vfs.walk instead of os.walk "dirpath" in the tuple yielded by "vfs.walk()" is relative one from the root of specified vfs, and absolute path in the warning message is composed by "vfs.join()". On the other hand, target file "f" exists in "dirpath", and "reljoin()" is needed to unlink "f" by "vfs.unlink()".

vfs: add walk To eliminate "path prefix" (= "the root of vfs") part from "dirpath" yielded by "os.walk()" correctly, "path prefix" should have "os.sep" at the end of own string, but it isn't easy to ensure it, because: - examination by "path.endswith(os.sep)" isn't portable Some problematic encodings use 0x5c (= "os.sep" on Windows) as the tail byte of some multi-byte characters. - "os.path.join(path, '')" isn't portable With Python 2.7.9, this invocation doesn't add "os.sep" at the end of UNC path (see issue4557 for detail). Python 2.7.9 changed also behavior of "os.path.normpath()" (see *) and "os.path.splitdrive()" for UNC path. vfs root normpath splitdrive os.sep required =============== ============== =================== ============ z:\ z:\ z: + \ no z:\foo z:\foo z: + \foo yes z:\foo\ z:\foo z: + \foo yes [before Python 2.7.9] \\foo\bar \\foo\bar '' + \\foo\bar yes \\foo\bar\ \\foo\bar (*) '' + \\foo\bar yes \\foo\bar\baz \\foo\bar\baz '' + \\foo\bar\baz yes \\foo\bar\baz\ \\foo\bar\baz '' + \\foo\bar\baz yes [Python 2.7.9] \\foo\bar \\foo\bar \\foo\bar + '' yes \\foo\bar\ \\foo\bar\ (*) \\foo\bar + \ no \\foo\bar\baz \\foo\bar\baz \\foo\bar + \baz yes \\foo\bar\baz\ \\foo\bar\baz \\foo\bar + \baz yes If it is ensured that "normpath()"-ed vfs root is passed to "splitdrive()", adding "os.sep" is required only when "path" part of "splitdrive()" result isn't "os.sep" itself. This is just what "pathutil.nameasprefix()" examines. This patch applies "os.path.normpath()" on "self.join(None)" explicitly, because it isn't ensured that vfs root is already normalized: vfs itself is constructed with "realpath=False" (= avoid normalizing in "vfs.__init__()") in many code paths. This normalization should be much cheaper than subsequent file I/O for directory traversal.

subrepo: pass wvfs to _sanitize instead of absolute path to a subrepository As a preparation for vfs migration of "_sanitize()", this patch passes "wvfs" to "_sanitize()" and use "wvfs.base" instead of absolute path to a subrepository.

repoview: move function for computing filtered hash An upcoming patch will establish per-filter tags caches. We'll want to use the same cache validation logic as the branch cache. Prepare for that by moving the logic for computing a filtered view hash to somewhere central.

revset: don't import discovery at module level discovery.py imports a lot of the world. Pierre-Yves told me to move it to a function-level import to avoid an import cycle in a future patch.

transaction: add missing newline to message Add a missing newline to the "journal was created by a different version of Mercurial" message.

cat: disable optimization of single file case for workingctx It crashes because workingctx has no manifest.

committablectx: override manifestnode() to return None wctx.manifestnode() crashed before because it has no _changeset. Instead of crashing, just return None like wctx.node().

cat: use ctx.manifestnode() in place of ctx._changeset[0]

changegroup: flush the ui stdio buffers after adding a changegroup This eliminates the following test failure on Windows, as well as a similar one in evolve's test-wireproto.t. See the previous patch for details on the problem. --- e:/Projects/hg/tests/test-init.t +++ e:/Projects/hg/tests/test-init.t.err @@ -216,10 +216,10 @@ * test 0:08b9e9f63b32 $ hg clone -e "python \"$TESTDIR/dummyssh\"" local ssh://user@dummy/remote-bookmarks searching for changes + exporting bookmark test remote: adding changesets remote: adding manifests remote: adding file changes remote: added 1 changesets with 1 changes to 1 files - exporting bookmark test $ hg -R remote-bookmarks bookmarks test 0:08b9e9f63b32

hook: forcibly flush stderr for Windows test stability There are a handful of SSH related test failures on Windows. --- c:/Users/Matt/Projects/hg/tests/test-bundle2-exchange.t +++ c:/Users/Matt/Projects/hg/tests/test-bundle2-exchange.t.err @@ -305,16 +305,16 @@ remote: added 1 changesets with 1 changes to 1 files remote: 1 new obsolescence markers updating bookmark book_5fdd + pre-close-tip:02de42196ebe draft book_02de + postclose-tip:02de42196ebe draft book_02de + txnclose hook: HG_SOURCE=push-response HG_TXNNAME=push-response + ssh://user@dummy/other HG_URL=ssh://user@dummy/other remote: pre-close-tip:5fddd98957c8 draft book_5fdd remote: pushkey: lock state after "bookmarks" remote: lock: free remote: wlock: free remote: postclose-tip:5fddd98957c8 draft book_5fdd remote: txnclose hook: (env vars truncated) - pre-close-tip:02de42196ebe draft book_02de - postclose-tip:02de42196ebe draft book_02de - txnclose hook: HG_SOURCE=push-response HG_TXNNAME=push-response - ssh://user@dummy/other HG_URL=ssh://user@dummy/other $ hg -R other log -G o 6:5fddd98957c8 draft Nicolas Dumazet <...> book_5fdd C | --- c:/Users/Matt/Projects/hg/tests/test-ssh.t +++ c:/Users/Matt/Projects/hg/tests/test-ssh.t.err @@ -438,12 +438,12 @@ $ hg push pushing to ssh://user@dummy/remote searching for changes + local stdout remote: adding changesets remote: adding manifests remote: adding file changes remote: added 1 changesets with 1 changes to 1 files - remote: KABOOM - local stdout + remote: KABOOM\r (esc) $ cd .. What is happening is that no data is available in 'sshpeer.pipee' while the command is executing. As the command completes, local output is printed, and then sshpeer.cleanup() is called. When it calls 'self.pipeo.close()', the child process is shutdown, flushing stderr. As an experiment, I printed a line to stdout and another to stderr instead this flush(). The stdout data was immediately available to the hg client, and none of the stderr data was until the child exited. At that point, pipee has all of the buffered data, and it is read out and printed before the pipe is closed in sshpeer.cleanup(). This is probably a known issue, since ui.write_err() mentions that stderr may be buffered, and also flushes stderr. It would be nice if there was a more general fix (there is one more test that fails), but I'm not sure what it is. I've seen (ancient) references [1] to setvbuf() "crashing spectacularly" on some systems if any I/O has been done already, so it seems worth avoiding. https://sourceware.org/ml/gdb-patches/2013-08/msg00422.html [1] https://groups.google.com/forum/#!msg/comp.lang.python/JT8LiYzYDEY/Qg9d1HwyjScJ

json: implement {fileannotate} template

json: implement {comparison} template Similar to {filediff}, we abbreviate some property names to cut down on string bloat.

json: implement {filediff} template Single letter properties are used to keep payload size down, as diff representation can be quite large and longer property names can create a lot of extra work for parsers. Rename is not yet captured. This can be done in a follow-up.

hgweb: expose raw line numbers to templates Surpringly, the templates didn't receive an unmodified version of the line numbers. Expose it to make implementing the JSON templates easier. In theory, we could post-process an existing template variable. But extra string manipulation seems quite wasteful, especially on items that could occur hundreds or even thousands of times in output.

revert: stop marking files clean after interactive revert (issue4592) The goal of 'hg revert --interactive' is usually to keep some change in the revert file, so the files -must-not- be marked as clean. We want the status logic to do its usual job here. For unclear reasons (probably timing related), I was unable to build an automated test that reproduced issue4592 but manual testing shows this is fixed.

lazymanifest: prevent leak when updating an entry more than once __setitem__ on the lazymanifest C type wasn't checking to see if a line had previously been malloced before replacing it, leading to leaks if files got updated multiple times in the course of a task. I was able to reproduce the leak with this change to test-manifest.py: diff --git a/tests/test-manifest.py b/tests/test-manifest.py --- a/tests/test-manifest.py +++ b/tests/test-manifest.py @@ -456,6 +456,16 @@ class basemanifesttests(object): ['a/b/c/bar.txt', 'a/b/c/foo.txt', 'a/b/d/ten.txt'], m2.keys()) + def testManifestSetItem(self): + m = self.parsemanifest('') + for x in range(3): + m['file%d' % x] = BIN_HASH_1 + for x in range(3): + m['file%d' % x] = BIN_HASH_2 + import time + time.sleep(4) + + along with the commands: $ make local $ PYTHONPATH=. SILENT_BE_NOISY=1 python tests/test-manifest.py testmanifestdict.testManifestSetItem & $ sleep 4 $ leaks $(jobs -p | tee /dev/stderr | awk '{print $3}') $ wait in an interactive shell on OS X. As far as I can tell, it had to be an interactive shell so that I could get the pid of the test run using the jobs builtin. Prior to this change, I was leaking several strings, and after this change leaks reports no leaks. I thought there was a bug filed for this in bugzilla, but I can't find it either in bugzilla or by searching my email.

strip: properly clear resolve state with --keep (issue4593) Normal updates automatically clean up the resolve state, but strip --keep does a "manual" update that bypasses the normal machinery. This adds a mergestate reset.

revsets: more informative syntax error message I came across a case where an internal command was using a revset that I didn't immediately pass in and it was difficult to debug what was going wrong with the revset. This prints out the revset and informs the user that the error is with a rebset so it should be more obvious what and where the error is.

revsets: show current revset abort behavior

tests: move blackbox testing of tags to test-tags.t We're going to refactor tags cache shortly. It is easier to test the blackbox logging if these tests are in test-tags.t.

tests: move mock blackbox extension into own file Having all blackbox log testing in test-blackbox.t isn't scalable. Move the mock blackbox extension into its own file so we can start to move blackbox logging into other tests.

shelve: acquire lock in the right order Text book says that 'wlock' should be acquired before 'lock'. Caught through developer warning.

check-commit: be more picky about detection of wrong bug tag The check-commit script search for "bug" withing bracket and ask people to use (issueXXXX) instead. The test was too wide and matching any "(+b+u+g"sequence.

discovery: don't compute allfuturecommon when it won't be used In repos with many changesets, the computation of allfuturecommon can take a significant amount of time. Since it's only used if there's an obsstore, don't compute it otherwise.

treemanifest: separate flags for trees in memory and trees on disk When we start writing tree manifests with one manifest revlog per directory, it will still be nice to be able to run tests using tree manifests in memory but writing to a flat manifest to a single revlog. Let's break the current '_usetreemanifest' flag on the revlog into '_treeinmem' and '_treeondisk'. Both are populated from the same config, but after this change, one can temporarily hard-code _treeinmem=True to see that tests still pass.

manifestdict: drop empty-string argument when creating empty manifest manifestdict() creates an empty manifestdict, so let's consistently use that instead of explicitly parsing an empty string (which does result in an empty manifest).

lazymanifest: fix memory leak in lmiter_iterentriesnext() after 3d485727e45e

revert: do not requires '--all' if '--interative' is present The '--all' option have been introduced in 890e285c52a1 (August 2006), most probably to prevent user shooting themselves in the foot. As the record process will let you, view and select the set of files and change you want to revert, I feel like the '--all' flag is superfluous in the '--interactive' case.

bundle2: drop the experimental hooks The series at e9ede9b4c2f8::8e13cc0825f1 introduced generic transaction level hooking. This makes the experimental bundle2 specific hooks redundant, we drop them.

bundle2: advertise bundle2 by default That way, any new server will be ready to accept bundle2 payload. The decision for the client to use it is still off by default so this is not turning bundle2 everywhere. We introduce a new kill switch for this in case stuff goes wrong.

subrepo: convert the os.path references in git to vfs There are a handful of os.path references in the free functions at the top of the module that will be trickier to remove.

subrepo: use vfs.removedirs instead of os.removedirs This patch also removes useless composing absolute path by "repo.wjoin()".

vfs: add removedirs

util: add removedirs as platform depending function According to fa901423ac23 introducing "windows._removedirs()": If a hg repository including working directory is a reparse point (directory symlinked or a junction point), then using os.removedirs will remove the reparse point erroneously. "windows._removedirs()" should be used instead of "os.removedirs()" on Windows. This patch adds "removedirs" as platform depending function to replace "os.removedirs()" invocations for portability and safety

subrepo: use vfs.unlink instead of os.remove This patch also removes useless composing absolute path by "os.path.join()".

subrepo: use vfs.rmtree instead of shutil.rmtree This patch also removes useless "shutil" import.

vfs: add rmtree This duplicates "onerror()" function from "svnsubrepo.remove()" for equivalence of replacing in subsequent patch. This "onerror()" function for "shutil.rmtree()" was introduced by 92b0d669637f, which avoids failure of removing svn repository on Windows.

subrepo: use vfs.readdir instead of os.listdir to avoid expensive stat calls "kind" information given from "vfs.readdir()" makes expensive stat calls "os.path.isdir()" and "os.path.islink()" useless.

editor: prefer 'intermediate-source' extra to use for HGREVISION environment variable Revision 51930a7180bd ('graft: record intermediate grafts in extras') introduced 'intermediate-source' extra which refers to the closest graft source. As 'intermediate-source' extra provides more detailed information about the source changeset than 'source' one, it is better to prefer the first one to use as a value of HGREVISION environment variable for an editor.

bundle2: rename format, parts and config to final names It is finally time to freeze the bundle2 format! To do so we: - rename HG2Y to HG20, - drop "b2x:" prefix from all part names, - rename capability to "bundle2-exp" to "bundle2" - rename the hook flag from 'bundle2-exp' to 'bundle2'

manifestdict: extract condition for _intersectfiles() and use for walk() The condition on which manifestdict.matches() and manifestdict.walk() take the fast path of iterating over files instead of the manifest, is slightly different. Specifically, walk() does not take the fast path for exact matchers and it does not avoid taking the fast path when there are more than 100 files. Let's extract the condition so we don't have to maintain it in two places and so walk() can gain these two missing pieces of the condition (although there seems to be no current caller of walk() with an exact matcher).

manifestdict.walk: remove now-redundant check for match.files() When checking whether we can take the fast path of iterating over matcher files instead of manifest files, we check whether match.files() is non-empty. However, now that return early for match.always(), it can only be empty when there are only include/exclude patterns, but in that case anypats() will be True, so it's already covered. This makes manifestdict.walk() more similar to manifestdict.matches().

manifest.walk: special-case match.always() for speed This cuts down the run time of hg files -r . > /dev/null from ~0.850s to ~0.780s on the Firefox repo. Note that manifest.matches() already has the corresponding optimization.

manifest.walk: use return instead of StopIteration in generator Using "return" within a generator is supposedly more Pythonic than raising StopIteration.

archive: look for first visible revision to build repo identity (issue4591) No test for the case where all revisions are hidden because "archive" command aborts if the target revision is null.

largefiles: use common function to build content of .hg_archival.txt This fixes the missing "changessincelatesttag" field introduced by 242d11819c6c.

test-largefiles: verify content of .hg_archival.txt

archive: extract metadata() closure to module-level function This function will be reused in largefiles.

archive: use ctx object consistently to build meta data

templatekw: have {manifest} use ctx.manifestnode() for consistency changeset_printer was updated at f57640bf10d4 to not access changeset by index.

subrepo: use vfs.reljoin instead of os.path.join

subrepo: inline reporelpath into abstractsubrepo._relpath to centralize logic "reporelpath()" is referred only from "abstractsubrepo._relpath()".

subrepo: add _relpath field to centralize subrelpath logic This patch adds propertycache-ed "_relpath" field to "abstractsubrepo", to centralize "subrelpath" logic into it. Now, "subrelpath()" can always return "_relpath" field of the specified subrepo object, because it is ensured that subrepo object has it. To reduce changes in this patch, "subrelpath()" itself is still kept, even though it seems to be redundant. This is also a part of eliminating "os.path.*" API invocations for "Windows UTF-8 Plan".

subrepo: add wvfs field to access the working directory via vfs This patch doesn't create vfs object in "abstractsubrepo.__init__()" but adds propertycache-ed "wvfs" field, because the latter can: - delay vfs instantiation until it is actually needed - allow to use "hgsubrepo._repo.wvfs" as "wvfs" "hgsubrepo._repo" is initialized after "abstractsubrepo.__init__()" invocation, and passing "hgsubrepo._repo.wvfs" to "abstractsubrepo.__init__()" is difficult.

subrepo: change arguments of abstractsubrepo.__init__ (API) This patch passes "ctx" and "path" instead of "ui" to "abstractsubrepo.__init__()" and stores them as "_ctx" and "_path" to use them in subsequent patches. This also removes redundant field initializations in the constructor of classes derived from "abstractsubrepo".

treemanifest: optimize treemanifest._walk() to skip directories This makes treemanifest.walk() not visit submanifests that are known not to have any matching files. It does this by calling match.visitdir() on submanifests as it walks. This change also updates largefiles to be able to work with this new behavior in treemanifests. It overrides match.visitdir(), the function that dictates how walk() and matches() skip over directories. The greatest speed improvements are seen with narrower scopes. For example, this commit speeds up the following command on the Mozilla repo from 1.14s to 1.02s: hg files -r . dom/apps/ Whereas with a wider scope, dom/, the speed only improves from 1.21s to 1.13s. As with similar a similar optimization to treemanifest.matches(), this change will bring out even bigger performance improvements once treemanifests are loaded lazily. Once that happens, we won't just skip over looking at submanifests, but we'll skip even loading them.

import-checker: force 'fcntl', 'grp', 'pwd', and 'termios' to stdlib modules These are Unix only, and caused these additional warnings on Windows if they aren't hardcoded as stdlib: mercurial/posix.py mixed imports stdlib: errno, getpass, os, socket, stat, sys, tempfile relative: grp, pwd, unicodedata mercurial/posix.py mixed imports stdlib: re relative: fcntl mercurial/posix.py mixed imports stdlib: array relative: termios Additionally, this was missing on Windows: mercurial/crecord.py mixed imports stdlib: fcntl, termios relative: curses https://docs.python.org/2/library/fcntl.html https://docs.python.org/2/library/grp.html https://docs.python.org/2/library/pwd.html https://docs.python.org/2/library/termios.html

import-checker: allow *.pyd based stdlib modules These are Windows dlls, and eliminate the following import check diffs that are not on Unix: mercurial/changegroup.py mixed imports stdlib: os, struct, tempfile, zlib relative: bz2 mercurial/encoding.py mixed imports stdlib: locale, os relative: unicodedata

manifest.walk: join nested if-conditions This makes it more closely match the similar condition in manifestdict.matches().

manifestdict: inline _intersectfiles() The _intersectfiles() method is only called from one place, it's pretty short, and its caller has to be aware when it's appropriate to call it (when the number of files in the matcher is not too large), so let's inline it.

manifestdict._intersectfiles: avoid one level of property indirection We have already bothered to extract "lm = self._lm", so let's use "lm" where possible.

manifestdict.matches: avoid name 'lm' for a not-lazymanifest

commands: add ui.statuscopies config knob statuscopies enables viewing of copies and moves in 'hg status' by default.

changelog: inline revlog.__contains__ in case it is used in hot loop Currently __contains__ is called only by "rev()" revset, but "x in cl" is a function that is likely to be used in hot loop. revlog.__contains__ is simple enough to duplicate to changelog, so just inline it.

bookmarks: show detailed status about outgoing bookmarks Before this patch, "hg outgoing -B" shows only difference of bookmarks between two repositories, and it isn't user friendly. This patch shows detailed status about outgoing bookmarks at "hg outgoing -B". To avoid breaking backward compatibility with other tool chains, this patch shows status, only if --verbose is specified,